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Abstract 


In commercial aviation, over 30-percent of all fatal accidents worldwide 
are categorized as Controlled Flight Into Terrain (CFIT) accidents, where a 
fully functioning airplane is inadvertently flown into the ground. The major 
hypothesis for a simulation experiment conducted at NASA Langley Research 
Center was that a Primary Flight Display (PFD) with synthetic terrain will 
improve pilots’ ability to detect and avoid potential CFITs compared to 
conventional instrumentation. All display conditions, including the baseline, 
contained a Terrain Awareness and Warning System (TAWS) and Vertical 
Situation Display (VSD) enhanced Navigation Display (ND). Each pilot flew 
twenty-two approach - departure maneuvers in Instrument Meteorological 
Conditions (IMC) to the terrain challenged Eagle County Regional Airport 
(EGE) in Colorado. For the final run, flight guidance cues were altered 
such that the departure path went into terrain. All pilots with a synthetic 
vision system (SVS) PFD (twelve of sixteen pilots) noticed and avoided the 
potential CFIT situation. The four pilots who flew the anomaly with the 
conventional baseline PFD configuration (which included a TAWS and VSD 
enhanced ND) had a CFIT event. Additionally, all the SVS display concepts 
enhanced the pilot’s situational awareness, decreased workload and 
improved flight technical error (FTE) compared to the baseline 
configuration. 


Summary 

Limited visibility is the single most critical factor affecting both the safety and capacity of worldwide aviation 
operations. In commercial aviation alone, over 30-percent of all fatal accidents worldwide are categorized as 
Controlled Flight Into Terrain (CFIT), where a mechanically sound and normal functioning airplane is 
inadvertently flown into the ground, water, or an obstacle, principally due to the lack of outside visual 
reference and situation awareness. The NASA Aviation Safety Program’s Synthetic Vision Systems (SVS) 
Project is developing technologies with practical applications that will eliminate low visibility conditions as a 
causal factor to civil aircraft accidents, as well as replicate the operational benefits of flight operations in 
unlimited ceiling and visibility-day conditions, regardless of the actual outside weather or lighting condition. 
The technologies will emphasize the cost-effective use of synthetic/enhanced- vision displays, worldwide 
navigation, terrain, obstruction, and airport databases, and Global Positioning System (GPS)-derived 
navigation to eliminate “visibility-induced” (lack of visibility) errors for all aircraft categories (transports, 
General Aviation, rotorcraft). A major thrust of the SVS Project is to develop and demonstrate affordable, 
certifiable display configurations which provide intuitive out-the-window terrain and obstacle information, 
including guidance information for precision navigation and obstacle/obstruction avoidance for Commercial 
and Business aircraft. 

To date, much of the SVS research has focused on introducing SVS display technology into as many existing 
aircraft as possible by providing a retrofit approach. This approach employs existing head down display 
(HDD) capabilities for glass cockpits (cockpits already equipped with raster-capable HDD’s) and head-up 
display (HUD) capabilities for the other aircraft. Two major NASA flight tests and several simulator studies 
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have occurred for assessment and evaluation of the SVS developments and the retrofit approach. The HDD 
objective of these studies was to examine whether an SVS display could be retrofitted into an Electronic 
Flight Instrumentation System (EFIS) Size “A” (e.g., B-757-200) Electronic Attitude Direction Indicator 
(EADI) and Size “D” (e.g., B-777) Primary Flight Display (PFD). A Size “X” (8” x 10” effective display 
area) head-down display was also tested that may represent the display real estate available on current high- 
end buisness aircraft. The HUD objective was to examine the feasibility of the concept of retrofitting SVS 
display technology onto HUDs for aircraft without raster-capable HDDs. Although promising results were 
obtained for the SVS-HUD concept, two significant deficiencies were found in daylight HUD usage: illegible 
raster symbology renditions under some direct sunlight conditions and some reported terrain depiction 
illusions. Proposed solutions to these daylight HUD deficiencies have been identified. The feasibility of the 
concept of retrofitting SVS display technology with HUD’s was verified for nighttime operations. No 
significant deficiencies were found in nighttime HUD usage. Two terrain-texturing techniques were also 
evaluated during the research. One method of terrain texturing, generic texturing, involved the selection of 
terrain color based on absolute altitude. The other method of terrain texturing, photo-realistic texturing, 
employed full-color ortho-rectified aerial photographs draped over the elevation model. The results of those 
studies confirmed that an SVS display, with pilot-selectable field of view (FOV), could be incorporated as 
part of an EFIS suite and effectively replace an EADI or PFD. Regardless of HDD display size, and for both 
day and night HUD applications, pilots reported greater situation awareness and had lower flight technical 
error (FTE) while operating with the SVS displays compared to conventional displays. For both HDD and 
HUD applications, no significant performance effects were found between texturing techniques, although 
most of the pilots preferred the photo-realistic terrain texturing technique to the generic texturing technique. 

The current study used the above retrofit display factors in a full factorial simulation experiment as the 
backdrop for conducting a Tare-event’ display concept comparison experiment to directly address CFIT 
avoidance benefits, which have been advanced as a primary motivation for SVS displays (another primary 
motivation is to replicate the operational benefits of flight operations in unlimited ceiling and visibility-day 
conditions, regardless of the actual outside weather or lighting condition). To prevent test subjects in an 
experiment from expecting a problem, Tare event’ simulation techniques require many simulation trials to 
produce only a few trials containing the data of interest, the rare event that allows for the potential creation of 
an accident scenario. 

Sixteen subject pilots (15 commerical transport pilots and 1 NASA research pilot) participated in the Tare- 
event’ display concept comparison experiment recently completed in the Visual Imaging Simulator for 
Transport Aircraft Systems (VISTAS III) at Fangley Research Center. The Tare-event’ (a course anomaly) 
was imposed only once on each pilot (with only one display condition) at the conclusion of repeated 
exposures to Instrument Meteorological Conditions (IMC) approach and departure operations at a terrain 
challenged airport using both conventional (one) and synthetic vision (six) display concepts. All seven 
display concepts also included an enhanced Navigation Display (ND) incorporating both a Terrain Awareness 
and Warning System (TAWS) display and a Vertical Situation Display (VSD), which presented a vertical 
profile of terrain along track. Each pilot flew twenty-two approach - departure maneuvers in IMC to the 
terrain challenged Eagle County Regional Airport (EGE) in Colorado. For the final run, the flight guidance 
cues were altered such that the departure path went into the terrain. All pilots with an SVS enhanced PFD 
(twelve of sixteen pilots) noticed and avoided the potential CFIT situation. The four pilots who flew the 
anomaly with the baseline display configuration (which included a TAWS and VSD enhanced ND) had a 
CFIT event. Additionally, data metrics from the experiment revealed that all of the SVS display concepts 
enhanced the pilots’ situational awareness, decreased workload and improved FTE compared to the baseline 
display configuration, during the numerous nominal and the single anomalous operations. 
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Introduction 


Limited visibility is the single most critical factor affecting both the safety and capacity of worldwide aviation 
operations. In commercial aviation alone, over 30-percent of all fatal accidents worldwide are categorized as 
Controlled Flight Into Terrain (CFIT), where a mechanically sound and normal functioning airplane is 
inadvertently flown into the ground, water, or an obstacle, principally due to the lack of outside visual 
reference and situation awareness (Boeing, 1998; Williams et ah, 2001). The NASA Aviation Safety 
Program’s Synthetic Vision Systems (SVS) Project is developing technologies with practical applications that 
will eliminate low visibility conditions as a causal factor to civil aircraft accidents, as well as replicate the 
operational benefits of flight operations in unlimited ceiling and visibility-day conditions, regardless of the 
actual outside weather or lighting condition. The technologies will emphasize the cost-effective use of 
synthetic/enhanced-vision displays, worldwide navigation, terrain, obstruction, and airport databases, and 
Global Positioning System (GPS)-derived navigation to eliminate “visibility-induced” (lack of visibility) 
errors for all aircraft categories (transports, General Aviation, rotorcraft). A major thrust of the SVS Project 
is to develop and demonstrate affordable, certifiable display configurations which provide intuitive out-the- 
window terrain and obstacle information, including advanced pathway and guidance information for precision 
navigation and obstacle/obstruction avoidance for Commercial and Business aircraft. 

Better pilot situation awareness (SA) during low visibility conditions can be provided by SVS displays. New 
technological developments in navigation performance, low-cost attitude and heading reference systems, 
computational capabilities, and displays allow for the prospect of SVS displays for virtually all aircraft 
classes. SVS display concepts employ computer-generated terrain imagery, on-board databases, and precise 
position and navigational accuracy to create a three dimensional perspective presentation of the outside 
world, with necessary and sufficient information and realism, to enable operations equivalent to those of a 
bright, clear, sunny day regardless of the outside weather or lighting condition. 

Controlled Flight Into Terrain 

Aviation has been witness to rapid advancement in technologies that have significantly improved aviation 
safety. The development of attitude indicators, flight management systems (FMS), radio navigation aids, and 
instrument landing systems (ILS) have extended aircraft operations into weather conditions with reduced 
forward visibility. However, as Brooks (1997) has noted, “...while standard instrumentation has served us 
well, enabling aviation as we see it today, literally thousands of dead souls, victims of aviation catastrophe, 
offer mute and poignant testimony to its imperfections. The simple, elegant dream of soaring aloft visually , 
intuitively - bird-like - remain elusive (Italics added, p. 17). Pilots must cope within an alphanumeric ’filter 
of symbology 4 to achieve spatial awareness; something that has repeatedly met with deadly consequences.” 
The significant number of Part 121 CFIT accidents demonstrates how even competent and professional pilots 
can lose terrain awareness. 

Wiener (1977) defined a CFIT accident as “one in which an otherwise-serviceable aircraft, under control of 
the crew, is flown (unintentionally) into terrain, obstacles or water, with no prior awareness on the part of the 
crew of the impending collision.” Approximately 40% of all aircraft accidents are CFIT and the category 
accounts for 50% of all aircraft fatalities (Mathews, 1997). Khatwa and Roelen (1996) reported that 70% of 
CFIT accidents occurred during the descent, approach and landing phases, while only 20% occurred during 
the enroute portion of a flight (Scott, 1996). To date, estimates are that over 20,000 people have lost their 
lives in CFIT accidents. 
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The crash of American Airlines (AA) 965 that occurred in 1995 in Cali, Columbia is often discussed as a 
classic example of how CFIT accidents happen. The Boeing 757-223 struck mountainous terrain during a 
VOR (Very high frequency Omnidirectional Range) DME (Distance Measuring Equipment) approach under 
instrument flight rules and visual meteorological conditions. The Aeronautica Civil of the Republic of 
Columbia cited several probable causes of the accident, including “the lack of situation awareness of the 
flight crew regarding vertical navigation, proximity to terrain and the relative location of critical radio aids.” 

CFIT accidents are often the outcome of a chain of events that contribute to a loss of situation awareness and 
human error, and the Cali accident was not an exception. Simmons’ (1998) identified 27 errors committed by 
the flight crew and air traffic control that together formed the latent and active failures for the CFIT to occur 
(Reason, 1990). The accident chain began with a “cleared to Cali VOR” clearance that was ambiguous and 
incorrectly read back, which prompted the Captain (pilot not flying) to put AA 965 on a direct course to Cali. 
Proceeding direct to Cali normally would not pose a terrain concern, but the action caused the Tulua (ULQ) 
VOR to be dropped from the active LEGS page in the FMS Control Display Unit (CDU). Later, when the 
Captain accepted an improper clearance to Rozo Non-Directional Beacon (NDB), he entered “R” in the direct 
intercept page. Unfortunately, there were two pages of SELECT DESIRED WPT (waypoint) or 12 NDBs 
named “R” and the Captain may have erroneously assumed that the closest R was Rozo. However, Columbia 
has two NDBs with an R identifier and the same frequency of 274 kHz, and the action selected the Romeo 
NDB instead. Because the Captain did not confirm the path on the FMS plan display, the flight crew was 
unaware they changed the aircraft path to an NDB located 132 nautical miles away. Although AA 965 was 
cleared for the approach, the Aeronautical Information Manual (AIM) (5-4-7-b) states that, “for IFR 
(Instrument Flight Rules) operations (Federal Aviation Regulation (FAR) Part 91.177), maintain the last- 
assigned altitude unless a different altitude is assigned by Air Traffic Control (ATC), or until the aircraft is 
established on a segment of a published route or IAP (Instrument Approach Procedure).” In other words, 
“cleared for approach” does not mean, “cleared to descend” and AA 965 descended from 15,000 feet before it 
was established on the 202-degree radial of Tulua. 

The decision of AA 965 flight crew to descend despite not being established on the approach segment is 
incidentally the same type of error responsible for the Trans World Airlines (TWA) 514 CFIT accident at 
Washington Dulles International in 1974. In that accident, the Boeing 727 was cleared for VOR DME 
Runway 12 approach and the flight crew initiated a descent to initial approach altitude (1800 feet) before the 
aircraft reached the appropriate approach segment resulting in a CFIT twenty-five nautical miles northwest of 
the airport. The TWA 514 accident prompted the installation of ground proximity warning systems (GPWS) 
on all Part 121 aircraft. However, as the AA 965 accident shows, the original GPWS was not as effective at 
prevention of CFIT as once hoped. Once AA 965 began the descent and failed to announce their intention to 
ATC, the aircraft fight path put the plane in a direct trajectory with the summit of El Deluvio (8,900 feet) 
thirty-three miles northeast of the Cali VOR. Subsequently, a GPWS warning was sounded but was 
ineffective because the flight crew failed to disengage the auto-throttle and advance power to maximum, 
coupled with a failure to retract the speed brakes (a procedure not required by GPWS training at the time of 
the accident). 

There is near unanimous consensus that, had the AA 965 flight crew had a graphic terrain presentation 
onboard, the flight crew would not had lost navigation awareness and would have realized that the aircraft 
was off course. This is seen in the Captain’s comments before the GPWS warning. The Captain remarked, 
“Ok, I’m getting it, seventeen seven. Just doesn’t look right on mine. I don’t know why.” A few seconds 
later, he commented, “it’s that [expletive] Tulua I’m not getting it for some reason....” Had AA 965 had an 
electronic display showing terrain relative to flight path, the confusion and concern of the Captain about the 
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approach would have been confirmed by the display. As it happened instead, moments later the GPWS 
sounded the “terrain, terrain, whoop, whoop, pull up ...” warning only twelve seconds before terrain impact 
resulting in the loss of 163 passengers with only four survivors. 

Terrain Awareness and Warning Systems 

The Flight Safety Foundation (FSF) CFIT Task Force reported that the absence of GPWS equipment or 
improper use of GPWS, and the use of “step-down” approach paths are associated with the majority of CFIT 
accidents. In fact, non-precision step-down approaches have a five times greater likelihood of CFIT 
occurrence than precision approaches. These conclusions prompted the recommendation of terrain awareness 
training programs and avoidance on non-precision approaches for large, transport category aircraft. In 
addition, the task force recommended the development of advanced technology including terrain databases, 
advanced GPWS, and enhanced and synthetic vision systems (Enders et al., 1996; Khatwa, & Roelen, 1996; 
Khatwa & Helmreich, 1999). The latter recommendation reflecting data showing that 40% of CFIT accidents 
involving aircraft with conventional GPWS received a late warning or improper pilot response during the 
years from 1988 to 1993; estimates of latency range from 10 to 15 seconds of warning (Flight International, 
1996; 1997). Another 16% of CFIT accidents received no warning at all. In addition, airlines have reported 
hundreds of false or nuisance warnings. These figures led the Federal Aviation Administration (FAA) 
Human Factors Team (SA-3; 1996) to recommend the replacement of conventional GPWS with earlier, more 
accurate indications and warnings of potential collisions with terrain; accurate position information; 
prediction algorithms for aircraft flight path; and terrain depiction on electronic displays (FAA, 1996). 
Similar recommendations came from the National Transportation Safety Board (NTSB) in response to the AA 
965 accident. The FAA (TAWS TSO C151A) and Joint Aviation Authorities (JAR OPS 1.665) have since 
mandated the installation of Class A Terrain Awareness and Warning Systems (TAWS) on all Part 121 
aircraft. 

TAWS is the generic term for warning systems that augments conventional GPWS through the presentation 
of terrain information on an electronic display derived from a worldwide terrain, obstacle, airport, and 
envelope modulation database. Currently, there are two commercially available TAWS for commercial 
aircraft: Enhanced GPWS (E-GPWS) and T2CAS. The E-GPWS incorporates the functions of the 
conventional GPWS, which has seven warning modes: 

Excessive Descent Rate: “Sinkrate”, “Pull Up” 

Excessive Terrain Closure Rate: “Terrain... Terrain”, “Pull Up” 

Altitude Loss After Takeoff: “Don’t Sink”, “Don’t Sink” 

Unsafe Terrain Clearance: “Too Low Terrain”, “Too Low Gear”, “Too Low Flaps” 

Excessive Deviation Below Glideslope: “Glideslope” 

Advisory Callouts: “Bank Angle”, “Minimums”, “50, 40, 30...” 

Windshear Alert: “Caution Windshear”, “Windshear ... Windshear” 
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The E-GPWS (Honeywell MK V & MK VII) adds to these modes several “enhanced” features including 
envelope modulation, enhanced terrain and obstacle detection, terrain and alerting display (TAD), terrain 
ahead alerting, “peaks” mode, aural declutter, geometric altitude, and runway field and terrain floor clearance 
(Breen, 2001; Honeywell, 2000). These additional features provide a 60 second caution alert and presents 
terrain information to the pilot for terrain less than 2000 feet below the aircraft or within 400 feet vertical of 
the nearest runway elevation. The E-GPWS also can display a “peak” digital value of the highest and lowest 
terrain/obstacle elevation. 

The second TAWS, T2CAS, combines traffic avoidance (Aircraft Collision Avoidance System; ACAS) with 
a ground collision avoidance system (GCAS). The GCAS provides similar features as the E-GPWS but uses 
a climb rate model that “looks” 132 seconds ahead and 90 degrees into a descent, turn, or climb to ensure 
terrain separation. Conflicts are predicted when aircraft position is compared to digital terrain and airport 
information, and alerts are generated when predicted flight path will intersect threatening terrain. The system 
then uses the climb rate model to project actual escape trajectories as part of the warning algorithm, which 
provides both vertical and lateral escape maneuver capability (ACSS, 2003). 

Theoretical Foundations of Synthetic Vision 

Research has documented the safety benefits of TAWS (e.g., de Muynck, & Khatwa, 1999; Kutchar & 
Hansman, 1993), and Ladkin (1997) asserted that there is near unanimity that TAWS has improved aviation 
safety and reduced the incidences of CFIT. However, TAWS still follows the “warn-act” model and, 
therefore, requires the flight crew to be reactive rather than proactive. The technology provides a warning 
when theoretically the flight crew has already lost spatial and situation awareness and must then perform an 
escape maneuver. As Moroze et al. (1999) describe, the strategy may not be optimal given the time required 
to adequately encode and assess the situation. The Society of Automotive Engineers (SAE) Subcommittee G- 
10V (1995; SAE ARD50062) echoes these concerns in detailing several human factors issues associated with 
terrain separation assurance displays. They stated that, “...to be effective, Terrain Separation Assurance 
displays must be designed to facilitate immediate interpretation, to minimize mental workload, ...and to 
minimize the potential for interpretation errors” (p. 54, Issue No. 6.7-6). What is needed is an intuitive 
system that improves pilot situation awareness with respect to spatial orientation in terms of terrain and flight 
path, and does not require the pilot to divert visual attention and cognitive resources away from possible 
external events and primary flight reference; that is, to provide a human-centered technology that can help 
prevent rather than just inform the flight crews of a potential collision with terrain (Snow et al., 1999). The 
approach requires an understanding and exploitation of the unique information processing capability of flight 
crews and a design of the technology and interface to accommodate perceptual and cognitive capabilities of 
the pilots - the difference between a “natural” and a “coded” display. 

Theunissen (1997) discussed the concept of natural versus coded information. Natural information implies 
that the method of information acquisition by the pilot is similar to that experienced in Visual Meteorological 
Conditions (VMC) by looking out the window. Visual altitude judgment is an example of natural information 
acquisition. Coded information implies some type of information presentation to the pilot that requires 
interpretation to comprehend the actual value. An example of coded information is digital radio altitude. 
Theunnissen noted that it is very important to give the pilot information required to maintain situation 
awareness in low-visibility conditions and that natural information presentation is intuitive and able to be 
perceived in a much more rapid manner than coded information. SVS displays provide a natural presentation 
of the outside world with information that is intuitive and easy to process. Essentially, it provides a “picture” 
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of the outside world, rather than disparate pieces of alphanumeric information, and best supports humans' 
natural acquisition and encoding of the world. 


Synthetic Vision Display Concepts 

A major thrust of the SVS project involves the development and demonstration of affordable, certifiable 
display configurations which provide intuitive out-the-window terrain and obstacle information, including 
advanced pathway and guidance information for precision navigation and obstacle/obstruction avoidance, for 
Commercial and Business aircraft. In addition to forward-fit applications, a path to retrofit this technology 
into today’s transport aircraft fleet is also necessary to achieve the desired safety benefits, since 66% of 
today’s transport aircraft fleet is equipped with only electro-mechanical cockpit instrumentation. 

NASA’s SVS concept (Figure 1) provides a real-time, unobscured synthetic view of the world for the pilot. 
The display is generated by visually rendering an on-board terrain database (with additional airport and 
obstacle database information as necessary) using precise position and navigation data obtained through GPS 
(Global Positioning System) data, with augmentation possibly from differential correction sources such as 
Local Area Augmentation Systems (LAAS) and Wide Area Augmentation Systems (WAAS), as well as 
blending from on-board Inertial Navigation System (INS) information. The SVS display concept includes the 
intuitive display of intended flight path using a tunnel or pathway-in-the-sky presentation. When coupled 
with a synthetic view of the outside world, the spatically integrated depiction of the intended flight path and 
its relation to the world provides an intuitive, easily interpretable display of flight critical information for the 
pilot. Active imaging sensors (e.g., Forward Looking Infra Red (FLIR), millimeter wave (MW) radar), real- 
time hazard information (e.g., weather and wake vortices), and traffic information as provided by Traffic 
Alert and Collision Avoidance System (TCAS), Automatic Dependent Surveillance - Broadcast (ADS-B) and 
Traffic Information Services - Broadcast (TIS-B) can additionally enhance this synthetic vision display 
concept (SVDC). Although the display representation to the pilot is synthetically derived, object detection 
and integrity monitoring functions are envisioned to ensure sufficient accuracy and reliability for certification. 
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Figure 1. Synthetic Vision System concept. 


Retrofit Approach 

To date, much of the NASA SVS research has focused on introducing SVS display technology into as many 
existing aircraft as possible by providing a retrofit approach. This approach employs existing head down 
display (HDD) capabilities for glass cockpits (cockpits already equipped with raster-capable HDD’s) and 
head-up display (HUD) capabilities for the other aircraft. Two major NASA flight tests have occurred for 
assessment and evaluation of the SVS developments. Both flight tests have used the NASA/Langley (LaRC) 
Research Center’s Airborne Research Integrated Experimental System (ARIES) Boeing B-757-200 aircraft. 
The first flight test (Glaab et al., 2003) was flown Sept-Oct 2000 in nighttime operations at Dallas-Ft. Worth 
(FAA Identifier: DFW). The second flight test (Bailey et al., 2002A) was flown Aug-Sept 2001 in simulated 
daylight Instrument Meteorological Conditions (IMC) at Eagle County Regional Airport, CO (FAA 
Identifier: EGE). 

The objective of these tests and past simulator studies was to examine whether an SVS display could be 
retrofitted into an Electronic Flight Instrumentation System (EFIS) Size “A” (e.g., B-757-200) Electronic 
Attitude Direction Indicator (EADI) and Size “D” (e.g., B-777) Primary Flight Display (PFD). In addition to 
these display sizes, both a Size “X” head-down display and an SVS HUD concept were evaluated. The Size 
X display represented the test case of display real estate that may be available on current high-end business 
aircraft. Each of the display size variations of the SVS HDD concepts evaluated included a pilot-selectable 
field of view (FOV) feature to address the fixed display size limitations. The HUD objective was to examine 
the feasibility of the concept of retrofitting SVS display technology onto HUDs for aircraft without raster- 
capable HDDs. Although promising results were obtained for the SVS-HUD concept, two significant 
deficiencies were found in daylight HUD usage: illegible raster symbology renditions under some direct 
sunlight conditions and some reported terrain depiction illusions. Proposed solutions to these daylight HUD 
deficiencies have been presented by Bailey (2002B). The feasibility of the concept of retrofitting SVS 
display technology with HUD’s was verified for nighttime operations. No significant deficiencies were found 
in nighttime HUD usage. Two terrain-texturing techniques were also evaluated during the research. One 
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method of terrain texturing, generic texturing, involved the selection of terrain color based on absolute 
altitude. The other method of terrain texturing, photo-realistic texturing, employed full color ortho-rectified 
aerial photographs draped over the elevation model. The results of those studies confirmed that an SVS 
display, with pilot-selectable FOV, could be incorporated as part of an EFIS suite and effectively replace an 
EADI or PFD. Regardless of HDD display size, and for both day and night HUD applications, pilots reported 
greater situation awareness and had lower flight technical error (FTE) while operating with the SVS displays 
compared to conventional displays. For both HDD and HUD applications, no significant performance effects 
were found between texturing techniques, although most of the pilots preferred the photo-realistic terrain 
texturing technique to the generic texturing technique. 

Current Study 

To prevent test subjects in an experiment from expecting a problem, Tare-event’ simulation techniques 
require many simulation trials to produce only a few trials containing the data of interest, the rare event that 
allows for the potential creation of an accident scenario. The current study used the above retrofit display 
factors in a full factorial simulation experiment as the backdrop for conducting a Tare-event’ display concept 
comparison experiment to directly address CFIT avoidance benefits, which has been advanced as a primary 
motivation for SVS displays (another primary motivation is to replicate the operational benefits of flight 
operations in unlimited ceiling and visibility conditions, regardless of the outside weather). The Tare-event’ 
(a course anomaly) was imposed only once on each pilot (with only one display condition) at the conclusion 
of repeated exposures to IMC approach and departure operations at a terrain challenged airport (EGE) using 
both conventional (one) and synthetic vision (six) display concepts (two SVS HDD concepts and one HUD 
concept, with the two texturing techniques) without the anomaly. All seven display concepts also included an 
enhanced Navigation Display (ND) incorporating both a TAWS and a Vertical Situation Display (VSD, 
which presented a vertical profile of terrain along track). Both the TAWS (Muynck, & Khatwa, 1999) and 
VSD (Prevot, 1998) concepts have been researched independently. These displays by themselves have 
limitations; however, an integrated SVS solution involving perspective terrain on PFD, and TAWS and VSD 
on an enhanced ND, has the potential to provide the complete terrain depiction. 

Experiment Objectives 

The main purpose of the experiment was to directly address SVS CFIT avoidance benefits, but the data 
gathered during the backdrop retrofit display factors experiment was used to confirm/deny the results 
obtained in prior research efforts (Glaab et al., 2003, Bailey et al., 2002A) by: 

1. Determining whether an SVS can improve the pilot’s ability to detect a potential CFIT scenario compared to a 
baseline 757 EFIS display system with a TAWS and VSD enhanced ND. 

2. Confirming pilot usability / acceptability and Situational (Terrain) Awareness and workload benefits provided 
by a NASA SVDC Head-Up Display (SVDC-HUD) and the potential of the SVDC-HUD as a retrofit display 
solution for SVS concepts in Non-Glass cockpits (this experimental objective was constrained by the lack of 
high visual fidelity in simulated HUD presentations). 

3. Confirming pilot usability / acceptability and Situational (Terrain) Awareness and workload benefits provided 
by various-sized SVDC Head-Down Displays (SVDC-HDD). 

4. Confirming pilot usability / acceptability and Situational (Terrain) Awareness provided by photo-textured and 
generically-textured terrain database SVS concepts within NASA SVS concepts (HUD; Head-down Sizes A/B, 
X) in support of the retrofit display concept evaluation and SVDC development. 

5. Assessing closed-loop performance during manually flown landing approach and departure (go-around) 
maneuvers in a terrain-challenged operational environment with and without SVS display concepts, and 
quantifying performance with respect to required navigation performance (RNP) procedures. 
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Acronyms and Abbreviations 

AA American Airlines 

ACAS Aircraft Collision Avoidance System 

ADS-B Automatic Detection Surveillance - Broadcast 

AGL Above Ground Level 

AIM Aeronautical Information Manual 

ANOVA Analysis of Variance 

ARIES Airborne Research Integrated Experiment System 

ATC Air Traffic Control 

CDU Control Display Unit 

CFIT Controlled Flight Into Terrain 

COTS Commercial Off The Shelf 

DEM Digital Elevation Model 

DFW FAA airport identifier for Dallas/Fort Worth International Airport 

DME Distance Measuring Equipment 

EADI Electronic Attitude Direction Indicator 

EFIS Electronic Flight Instrumentation System 

EGE FAA airport identifier for Eagle County, Colorado Regional Airport 

E-GPWS Enhanced Ground Proximity Warning System 

FAA Federal Aviation Administration 

FAR Federal Aviation Regulation 

FLIR Forward-Looking Infra-Red 

FMS Flight Management System 

FOV Field of View 
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FPA 


Flight Path Angle 


FSF 

Flight Safety Foundation 

FTE 

Flight Technical Error 

GCAS 

Ground Collision Avoidance System 

GEOTIFF 

Georeferenced Tagged Image File Format 

GPS 

Global Positioning System 

GPWS 

Ground Proximity Warning System 

HDD 

Head-Down Display 

HUD 

Head-Up Display 

IAP 

Instrument Approach Procedure 

IFR 

Instrument Flight Rules 

ILS 

Instrument Landing Systems 

IMC 

Instrument Meteorological Conditions 

INS 

Inertial Navigation System 

LAAS 

Local Area Augmentation System 

LaRC 

Langley Research Center 

MASPS 

Minimum Aviation System Performance Standards 

MSL 

Mean Sea Level 

NASA 

National Aeronautics and Space Administration 

ND 

Navigation Display 

NDB 

Non-Directional Beacon 

NED 

National Elevation Dataset 

nmi 

nautical mile 

NOTEM 

Notice to Airmen 
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NTSB 

National Transportation Safety Board 

PC 

Personal Computer 

PFD 

Primary Flight Display 

RMI 

Radio Magnetic Indicator 

RMS 

Root Mean Square 

RNAV 

RNP Area Navigation 

RNP 

Required Navigation Performance 

SA 

Situation Awareness 

SAE 

Society of Automotive Engineers 

SA-SWORD 

Situational Awareness - Subjective Workload Dominance 

SNK 

Student-Newman-Keuls 

sv 

Synthetic Vision 

SVDC 

Synthetic Vision Display Concepts 

svs 

Synthetic Vision Systems 

TAD 

Terrain and Alerting Display 

TAWS 

Terrain Awareness and Warning System 

TCAS 

Traffic Collision and Avoidance System 

TIS-B 

Traffic Information Services - Broadcast 

TOGA 

T akeoff/Go-around 

TWA 

Trans World Airlines 

USGS 

United States Geological Survey 

UTM 

Universal Transverse Mercator 

VISTAS 

Visual Imaging Simulator for Transport Aircraft Systems 

VMC 

Visual Meteorological Conditions 
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VNAV 


VOR 

VSD 

WAAS 


Vertical navigation 

Very high frequency Omni-direction Radio 

Vertical Situation Display 

Wide Area Augmentation System 
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Methodology 

VISTAS III Simulation Facility 

The experiment was conducted in the Visual Imaging Simulator for Transport Aircraft Systems (VISTAS) III 
part task simulator at NASA Langley Research Center (Figure 2). The single pilot fixed-based simulator 
consists of a 144 degree by 30 degree out-the- window visual, a simulated HUD, a large field, reconfigurable 
screen for HDD and pilot input controls. A HUD is simulated by independently projecting HUD symbology 
(in monochrome green) on the out-the-window display immediately in front of the pilot. Though the 
simulated HUD is projected over 9 feet away from the pilot (one of the factors that limit the fidelity of the 
simulated HUD), the FOV for the simulated HUD is comparable to a standard HUD. For this experiment, the 
out-the-window scene was used only during training. 

The rudimentary pilot controls in the VISTAS III workstation are a left side-arm controller, left/right throttle 
controls with take-off/go-around (TOGA) button, rudder pedals, left/right toe brakes, landing gear switch and 
a Personal Computer (PC) track ball for display related pilot inputs. The track ball was used to select ND 
range scale and the SVS FOV. 



Figure 2. VISTAS III fixed-base pilot- in-the-loop workstation. 


Test Subjects 

A total of sixteen test subjects participated in this experiment. Fifteen subjects were airline pilots and one 
subject was a NASA researcher. The subjects from commercial airlines consisted of four captains and eleven 
first officers. The NASA researcher was an experienced Air Force transport pilot with no previous 
knowledge of SVS concepts. All subjects had HUD experience and all airline pilots had current commercial 
licenses. The subjects had an average 19.6 years of flying experience with an average 8200 hours logged 
(Appendix E, Table 9). The subjects were given a 30-minute briefing to explain the retrofit concept and the 
expected subject task. After the briefing, a 1.5-hour training session was conducted to familiarize the subjects 
with the aircraft model, display symbologies and controls. The Tare-event’ scenario was not discussed, 
although the pilot’s responsibility for maintaining terrain clearance at all times was stressed. Data collection 
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lasted approximately 4 hours followed by a 1 hour semi-structured interview. The entire session including 
lunch and breaks lasted approximately 8 hours. 

Terrain Databases 

The terrain databases used for this experiment covered a 95 nautical mile (nmi) by 95 nmi area centered at the 
EGE airport. Jeppesen provided the source elevation data for the EGE databases based on U.S. Geological 
Survey (USGS) National Elevation Dataset (NED). The delivered elevation data was 1-arcsecond (30 meter) 
in Digital Elevation Model (DEM) format, with a Universal Transverse Mercator (UTM) WGS84 projection 
and covered a 100 nmi square geographically centered about EGE. The accuracy of the source data was 
within 12 meters (90% of data) horizontal and 7 meters (90% of data) vertical. From this DEM, four real 
time rendering databases were created (Figure 3): 

1. SVS-HDD full color photo-realistic texture (Figure 3, top left) 

2. SVS-HDD full color elevation based (generic) texture (Figure 3, top right) 

3. SVS-HUD monochrome green photo-realistic texture (Figure 3, bottom left) 

4. SVS-HUD monochrome green elevation based (generic) texture (Figure 3, bottom right) 

The monochrome databases were created because HUD’s are monochrome green. The monochrome 
databases were designed for monochrome displays to ensure proper rendering (i.e., no holes in database due 
to lack of green color). Each EGE terrain database was built using commercially available terrain building 
software. The SV terrain databases were written to a UTM format and rendered using a commerically 
available scene graph software package. An EGE airport model was created using a commercial off the shelf 
(COTS) modeling software and placed into the SV database. 
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Color photo-realistic (top left), Green photo-realistic for HUD (bottom left), Color generic (top right), Green generic for 

HUD(bottom right). 

Figure 3. Four databases with symbology overlays used for the experiment. 


To create the full color photo-realistic terrain database, multi-resolution aerial imagery (Georeferenced 
Tagged Image File Format (GeoTIFF) ranging from 1 to 16 meters/pixel) was overlaid on the DEM database. 
The source data included three images sets of 1, 4, and 16 meter resolution (see Table 1), which were nested, 
with the highest resolutions centered about the EGE runway. The final database (Table 2) was created from 
the source data after a rendering trade off study. The trade off study maximized the amount of texturing to be 
rendered (most photo-realistic) while maintaining a 30 Hz update frame rate. A significant effort was made 
before the database build to color balance the various aerial images to produce a non-tiled, single brightness 
and contrast database appearance. To create the monochrome green photo-realistic database, full color aerial 
photographs were converted to a single green color with multiple shadings using a freeware image-editing 
tool. 
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Table 1. Photo-realistic image sources 


Image resolution 

Provider 

Format 

Area Coverage 
(centered on EGE) 

1 -meter per pixel 

NGS 

GeoTIFF WGS84 UTM Zone 13 

17.2 nmi (east/west) by 6.8 
nmi (north/south) 

4-meter per pixel 

ImageLinks 

GeoTIFF WGS84 UTM Zone 13 

29.7 nmi (east/west) by 32 
nmi (north/south) 

16-meter per pixel 

ImageLinks 

GeoTIFF WGS84 UTM Zone 13 

104.2 nmi (east/west) by 
104.2 nmi (north/south) 


T able 2. Final TerraPage database created from source da ta 


Image Resolution 

Coverage 

2-meter per pixel 

2.1 nmi by 0.7 nmi 

4-meter per pixel 

25 nmi by 25 nmi 

1 6-meter per pixel 

95 nmi by 95 nmi 


To create the generic textured terrain database, a color mapping technique (i.e., “elevation shading”) was 
developed. The color scheme chosen was similar to Aeronautical Chart legends with slight modification to 
show more contrast over the elevation range in the database. The colors ranged from greens (field elevation 
of EGE), to browns, to light tans, to off-white with the greens representing the lower elevations bands, and 
the off-white representing the highest elevation band. Twelve bands were used, segmented into 250-meter 
ranges. To create the monochrome green generic database, shades of green were used to represent elevation 
changes. The green color intensities associated with each elevation level varied in an incremental fashion 
from the lowest to highest level. Thus, no two elevation levels had the same green value. Main cultural 
features, such as railroads, roads, lakes, and rivers, were placed in the generic textured terrain databases. 

Experimental Test 

Evaluation Tasks 

The test subjects were asked to fly a circling visual approach to EGE runway 7 in IMC (Figure 4) with no 
out-the-window visibility. At 200 feet above ground level (AGL), a go around was declared and a departure 
path was followed. Both the approach and departure paths were slightly modified published approach and 
departure paths. The simulated aircraft used for this experiment was a Boeing 757. Both the approach and 
departure speeds were 140 knots. All scenarios were flown with light to moderate turbulence and no wind. 
For the approach part of the task, auto throttles were enabled, flaps were set to 30 degrees and the landing 
gear was down. At 200 ft AGL, a go around was executed (the pilot pressed the TOGA button) and the loss 
of one engine was simulated (for this study, both throttles were set to 40% power to simulate single engine 
power, a condition that entailed remaining low in the terrain during departure). Then, the pilot raised the 
landing gear and the flaps were set to 5 degrees. The task for the departure path was to follow a heading of 
050 degrees until the aircraft was at 6.8 DME from the Snow VOR. At the 6.8 DME point, the pilot was then 
to follow the 059 radial from the Snow (FAA designation SXW) VOR. The data collection run ended at the 
12.0 DME point from SXW. For the final data run of the experiment, the rare event scenario, the flight 
guidance was altered on the departure path (dotted path in Figure 4). Both the flight path and guidance cues 
directed the airplane into the side of a mountain peak. 
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TAWS aural alerts were demonstrated during the training session but were disabled for data runs. Because of 
the nature of the scenarios, the TAWS aural alerts were frequent and distracting even when following a 
known correct path (in the actual flight tests operations at EGE, it was necessary to disable the aural warning 
for the more rudimentary ground proximity warning system, GPWS, of the LaRC Boeing 757). It should be 
mentioned that the aural alerts would have occurred on all runs including the non CFIT scenarios, thus, the 
aural alert would not have aided in CFIT detection for these scenarios. 



Figure 4. Map view of the approach and departure (go around) task flown by the subject pilot. 

Display Conditions 

The subjects flew the same approach-departure task with three replications of each of the seven varying 
display presentations for a total of twenty-one data runs. The display variations were based on four tactical 
display types (Figure 5; baseline EADI, Size A (5” x 5.25”) SVS EADI , Size X SVS PFD, SVS HUD) and 
terrain texturing type (photo-realistic or generic). For all display presentations, the ND with TAWS and VSD 
was on a size B (4” x 6”) display. A Radio Magnetic Indicator (RMI) showing off path position for the Snow 
059 degree radial (SXW R-059), for use during the final part of the departure path, was also provided for all 
display presentations. For the SVS HUD display condition, the HDD was the baseline (EADI, TAWS and 
VSD enhanced ND) concept. The tactical display variations (Appendix C, Figures 35 - 41 presents all of the 
different display types) were: 

1. Baseline Size A (5” x 5.25”) EFIS 757 EADI display 

2. Size A SVS EADI display with color generic textured terrain 

3. Size A SVS EADI display with color photo-realistic textured terrain 

4. Size X (8”xl0”) SVS PFD with color generic textured terrain 

5. Size X SVS PFD with color photo-realistic textured terrain 

6. HUD (24 degree x 18 degree FOV) enhanced with SVS monochrome green generic textured terrain 

7. HUD enhanced with SVS monochrome green photo-realistic textured terrain 

The final run was a ‘rare-event’ scenario where the flight guidance directed the airplane into a mountain peak. 
The display type for the final run (Table 3) was varied across all sixteen test subjects. For the SVS displays 
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on the final run, only the photo-realistic database was used. Thus only four display variations were tested, 
with four replicates each. 


Table 3. Display type variation for the ‘rare-evenf CFIT data run 


Number of sub jects 

Display type for CFIT data run 

4 

Baseline EADI with TAWS and VSD enhanced ND, and 
with a RMI 

4 

Size A EADI with color photo-realistic textured terrain, 
with TAWS and VSD enhanced ND, and with an RMI 

4 

Size X PFD with color photo-realistic textured terrain, 
with TAWS and VSD enhanced ND, and with an RMI 

4 

HUD with monochrome photo-realistic textured terrain, 
with TAWS and VSD enhanced ND, and with an RMI 

Total 16 subjects 




and bottom right displays, while the bottom left display shows photo-realistic texturing. 


Figure 5. Display type Variations: Baseline (top left), Size A (top right), Size X (bottom left) and HUD (bottom right). 
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Experiment Matrix 


T able 4. Experiment data run matrix; three replications for each condi tion 


Repetitions 
per pilot 

Photo textured 
terrain 

Generic textured 
terrain 

Size A 

3 

3 

Size X 

3 

3 

HUD 

3 

3 


In addition to the 18 SVS data runs indicated above in Table 4, each pilot flew three EFIS baseline runs and a 
final CFIT run. Other than the last run being the ‘rare-event’ CFIT, the data runs were randomized for each 
pilot. 

For the final data run of the experiment session, a ‘rare-event’ CFIT scenario was presented to the subject 
pilot. The FMS was set such that the flight path and associated flight directors provided guidance into the 
terrain (dotted line path in Figure 4). The pilots were not briefed that the final run was any different than 
previous runs. In addition, the subjects were verbally instructed to fly the simulator as if it was an actual 
airplane with passengers on board and that they should take any necessary action to avoid the terrain. 
Information sources available to the pilot for potential detection of the anomaly included the TAWS and VSD 
terrain depictions, and a RMI showing off path position from the same path they had already flown twenty 
one times before (three times with the baseline). Twelve of the sixteen subject pilots also had terrain 
information on the SVS HDD (size A or X) or HUD. The photo-realistic textured database was used for all 
of the SVS ‘rare-event’ data runs. 

Measures 

Root mean square (RMS) metrics were computed from the measures for vertical and lateral deviations of the 
simulated B-757 from the defined approach path, and for lateral deviations from the defined departure path, 
along with speed error for the speed-on-pitch departure task. Pilots were asked structured questions after 
each run, and at the end of the day, a semi-structured interview was conducted, along with a Situation 
Awareness - Subjective Workload Dominance (SA-SWORD) questionnaire (Vidulich & Hughes, 1991). 
Additionally, subjects were given a “take home” questionnaire that was to be mailed back (all responses are 
in Appendix G). 

The approach and departure paths for the task were analyzed using a flight segment analysis approach (see 
Figure 6 and Table 5). Flight segments were used since the segments contain well-defined piloting tasks in 
which specific and clear performance expectations were given to the pilots, and control of statistical 
variability could thus be anticipated. FTE computations (which are one component of RNP calculations) 
were made from the recorded quantitative path error data for the task. These data were analyzed over the 
entire approach and departure path (all segments) using histogram analyses (Appendix C, Figures 46 - 49). 
For lateral path performance, the bin width definitions used were 0.05 nmi. For vertical path performance, 
the bin width definitions used were 50 feet. The number of occurrences in each bin was totaled and the total 
number of occurrences divided this total bin value over the entire approach or departure to determine the 
percentage of occurrences for each bin to form the histograms. 

For the rare-event CFIT data runs, impact or close proximity(within 100 feet) of the terrain was the main 
measure. For cases where the CFIT was detected, the distance from the SXW VOR, where the pilot noticed 
the potential CFIT, was recorded. The distance measure was converted to ‘time to impact’ based on distance 
to the impact point and ground speed. In addition, subjects were asked CFIT specific questions in the semi- 
structured interview. 
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Segment 7 


Segment 8 


Segment 5 



Figure 6. Segmentation of subject task for statistical analysis. 


Table 5. Flight Segment Definitions 


Segment # 

Task 

Starting Point 

Ending Point 

1 

Straight 4.43 degree descent 

5.3 DME SXW 

4.5 DME SXW 

2 

Halt descent at 8100 ft MSL 

4.5 DME SXW 

4.0 DME SXW 

3 

turn left to 220 degrees at 4.0 
DME; maintain 8100 ft MSL 

4.0 DME SXW 

Heading 220 degrees 

4 

Straight & level at 8100 ft 

9.2 nmi along path from 
touchdown zone of runway 7 

4.5 nmi from start of 
segment 

5 

Continuous right turn and 3.0 
degree descent 

4.7 nmi along path from 
touchdown zone of runway 7 

200 ft AGL 

Between 5 
& 6 

Subject over-flies runway, 
reconfigures airplane, 
establishes straight 50 degree 
heading at 140 knots; maintain 
climb with speed-on-pitch 

Missed approach at 200 ft AGL 

6000 ft from runway 7 
threshold 

6 

Straight 50 degree heading at 
140 knots; maintain climb with 
speed-on-pitch 

6000 ft from runway 7 threshold 

DME 6.8 SXW 

7 

Turn on to 059 degree radial 
from SXW 

DME 6.8 SXW 

On 059 radial of SXW 

8 

Straight 59 degree heading at 
140 knots; maintain climb with 
speed-on-pitch 

End of turn of segment 7 

12.0 DME SXW 
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Results 


Results from this experiment are presented for quantitative and qualitative data. Analyses of the rare-event 
CFIT data runs are presented first, followed by the pilot performance data, then the RNP-type data analyses, 
and finally the qualitative results are discussed. Data analysis of the quantitative path data (RMS vertical and 
lateral deviation) and speed maintenance (where appropriate) were done for the entire approach and the entire 
departure task, and by Task Segments for segments of moderate length (segments 1, 2, 3, and 7 were 
considered to be short transitions with potential for considerable statistical variability and thus were not 
analysed individually). The data were analyzed by univariate Analysis of Variance (ANOVA) across Subject, 
Display Type, and, where appropriate, Texture Type (SVS-only Display Types). Student-Newman-Keuls 
(SNK) tests (at a 5-percent significance level) of individual means were performed at appropriate stages in the 
analyses. Analyses of pilot performance data are presented first for the entire flight phase of the 
experiment, and then by individual segments, for both approach and the departure tasks. 

Quantitative Results 

The quantitative results presented are from statistical analyses of path and speed error performances (where 
appropriate). All analyses were conducted as full factorial within subject designs, and as expected for a 
precision task, the main effect of pilot variability was always highly significant for all measures. Only the 
statistically significant results of the other factors and second order interactions of interest are discussed. 

Rare Event CFIT Data Run 

For the CFIT scenario, twelve of the sixteen test subjects flew the CFIT scenario with an SVS enhanced PFD 
(four pilots with Size A, four pilots with Size X) or HUD (four pilots). All twelve pilots with the SVS display 
detected and avoided the potential CFIT. On average, pilots with an SVS display noticed the potential CFIT 
53.6 seconds (the standard deviation was 19.1 seconds) before impact with the terrain. Four of the sixteen 
pilots flew the CFIT scenario with the baseline display and all four pilots had a CFIT event. Three of the four 
pilots impacted the terrain while one passed within 58 feet of a mountain peak without awareness of any 
terrain separation problem. Even though the baseline concept had a TAWS and VSD enhanced ND, and an 
RMI, none of the subjects were aware of a CFIT event based on either the pilots’ maneuvers or post-run 
elicited comments. 

Figure 7 shows four snapshots taken at the same point in time for four Size A displays. The snapshots were 
taken on departure at 10.1 DME from SXW. The left two displays show the nominal departure and the right 
two displays show the CFIT departure. The SVS displays clearly show the surrounding terrain while the 
baseline EADI display provides no terrain awareness. Figure 8 shows the TAWS, VSD and RMI displays 
(which were common for all display concepts) at the same 10.1 DME point from SXW for the nominal (left) 
and CFIT (right) runs. The range shown for both the ND and the VSD is 20 nmi. For nominal runs, the 
departure task was aimed at a notch between two mountain peaks that could be seen on the SVS PFD. 
Because of the low resolution of the TAWS terrain database, the nominal TAWS showed a terrain warning 
(solid red, terrain impact within 30 seconds) over the notch region. The TAWS display correctly showed 
solid red for the CFIT scenario; however, the CFIT TAWS display is very similar in appearance to the 
nominal TAWS. The RMI was tuned to the 059 radial for SXW, and as seen in Figure 8, there is little 
difference between the nominal and CFIT RMI reading. Thus for the baseline display configuration, the 
display elements discussed so far provide few differences between the nominal and CFIT scenarios. 
However, the VSD information content is quite different. The peak shown on the VSD for the nominal run 
(left display of Figure 8) is 10 nmi (over four minutes) away; however, the run ends 1.9 nmi (at 12 DME from 
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SXW) from the present point, and so that peak is of no concern within the scenario. The VSD for the CFIT 
scenario (right display of Figure 8) clearly shows an imminent CFIT event (1 nmi, 26 seconds to impact). 



Figure 7. Baseline and SYS displays for nominal mn (two left) and baseline and SYS displays for CFIT mn (two right) 



Figure 8. TAWS, VSD and RMI for nominal mn (left) and CFIT mn (right) 


Aside from the rare event CFIT scenario, it should be noted that two CFIT’s occurred during nominal data 
mns (i.e., ground impact short of the mnway during nominal circling maneuvers in segment 5). Both 
unplanned CFIT’s occurred with the baseline display during the turn to final for two subjects, each during 
their first data runs with the baseline concept (note that the subject training included the baseline concept 
flying this task). With the baseline concept, a CFIT does not affect the simulator in any way; that is, the 
airplane continues to fly with no effects on its flight path. One of the unplanned CFIT’s was clearly a 
statistical outlier, which would increase the statistical variability enormously and was therefore excluded from 
the analysis. The second unplanned CFIT occurred because the turn to final was overshot somewhat and 
resulted in terrain collison with Snow mountain. However, the subject, unaware of the CFIT event, was able 
to quickly correct the overshoot and continue the data mn, and the RMS path error results were not atypical of 
baseline concept performances. 
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Inferences from the rare event CFIT Quantitative Results 

All of the subjects with an SVS display avoided the departure CFIT scenario. Subjects commented that they 
knew “something was wrong” from the presentation of the terrain on the PFD. It was clear on the SVS 
display that the velocity vector was overlaying vertical terrain directly in the path of the airplane, indicating 
an imminent terrain collision. Upon noticing a problem on the SVS PFD, most subjects crosschecked with 
the VSD to confirm there was a terrain collision imminent, and concluded that the guidance was erroneous. 
In addition, having 3-D perspective terrain gave awareness to the pilot for strategic escape maneuvering. The 
SVS displays gave subjects an intuitive view of the terrain, which is the theoretical basis for SVS CFIT 
prevention. As one subject stated, “Pilots don’t fly into mountains they can see.” 

Subjects who flew the CFIT scenario with the baseline concept, which included a TAWS and VSD enhanced 
ND, all had a CFIT event. For the baseline CFIT runs, the TAWS was very similar to the nominal runs which 
had annunciated warnings for all data runs. The CFIT path was offset by approximately 1 degree from the 
nominal run. The resulting erromeous indication on the RMI was not noticed by the subjects. The baseline 
EADI provides no terrain awareness; therefore the only indication of a terrain collision was on the VSD. This 
finding implies that in order for a VSD to aid in the prevention of a CFIT, sufficient training would be 
necessary so that pilots would always include the VSD in their scan. 

Entire Approach 

Baseline/S VS Displays - Analyses of variance were conducted on the main factors of Pilot and Display Type 
(baseline, Size A, Size X, HUD) for the path error performance measures with the following results: 


Approach RMS Path Error 



Figure 9. Entire approach path performance errors for display concepts without texturing effects. 

RMS lateral approach path error results (Figure 9) - Display Type (F(3,331)=98.754, p<.001) was highly 
significant for the measure of RMS lateral path error during the entire approach. Post hoc tests (using 
Student-Newman-Keuls, SNK, with a=.05) showed that with the baseline concept (mean=267 ft), 
significantly poorer tracking of the lateral path was obtained as compared to the three SVS concepts: Size A 
(mean=82 ft), Size X (mean=80 ft) and HUD (mean=61 ft). The comparison between the SVS concepts is 
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presented in the SVS Displays/Texture analysis below (the latter analysis has more statistical power to 
discriminate differences because the error term, even with fewer degrees of freedom, is so much smaller). 

RMS vertical approach path error results (Figure 9) - Display Type (F(3,331)=5.789, p<.001) was highly 
significant for the measure of RMS vertical path error during the entire approach. Post hoc test (using SNK 
with a=.05) showed that with the baseline concept (mean=106 ft), significantly poorer tracking of the vertical 
path was obtained as compared to the three SVS concepts: Size A (mean=69 ft), Size X (mean=63 ft) and 
HUD (mean=60 ft). The comparison between the SVS concepts is presented in the SVS Displays/Texture 
analysis below (the latter analysis has more statistical power to discriminate differences). 

SVS Displays/Texture - SVS-only Analyses of variance were conducted on the main factors of Pilot, SVS 
Display Type (Size A, Size X and HUD) and Texture Type for the path error performance measures with the 
following results: 

RMS lateral approach path error results (Figure 9) - SVS Display Type (F(2,279)=14.208, p<.001) was highly 
significant for the measure of RMS lateral path error during the entire approach. Post hoc tests (using SNK 
with a=.05) showed that performance with the HUD (mean=61 ft) was statistically better than with Size A 
(mean=82 ft) and Size X (mean=80 ft). The differences between Size A and Size X could not be 
discriminated. The improved performance was attributed to the HUD’s unity magnification factor, but the 
performance differences were not considered operationally significant for the entire approach maneuver 
measure. Neither terrain Texture Type nor the interaction between Display Type and Texture Type were 
significant for this measure. 

RMS vertical approach path error results (Figure 9) - SVS Display Type (F(2,279)=4.660, p<.010) was highly 
significant for the measure of RMS vertical path error during the entire approach. Post hoc tests (using SNK 
with a=.05) showed that with the Size A (mean=69 ft) concept, significantly poorer tracking of the vertical 
path was obtained as compared to the other two SVS concepts: Size X (mean=63 ft) and HUD (mean=60 ft). 
The differences between Size X and the HUD could not be discriminated. The decreased performance was 
attributed to the increased minification factor of the Size A display, although the effect was not considered to 
be operationally significant for the entire approach maneuver measure. Neither terrain Texture Type nor the 
interaction between Display Type and Texture Type were significant for this measure. 

Entire Departure 

Baseline/S VS Displays - Analyses of variance were conducted on the main factors of Pilot and Display Type 
(baseline, Size A, Size X, HUD) for the lateral path error and speed-on-pitch performance measures with the 
following results: 
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Figure 10. Entire departure path performance errors for display concepts without texturing effects. 

RMS lateral departure path error results (Figure 10) - Display Type (F(3,331)=34.008, p<.001) was highly 
significant for the measure of RMS lateral path error during the departure. Post hoc tests (using SNK with 
a=.05) showed that with the baseline concept (mean=577 ft), significantly poorer tracking of the lateral path 
was obtained as compared to the three SVS concepts: Size A (mean=212 ft), Size X (mean=166 ft) and HUD 
(mean=138 ft). The comparison between the SVS concepts is presented in the SVS Displays/Texture analysis 
below. 

RMS speed-on-pitch error results -Display Type (F(3,331)=.164, p>.05) was not significant as measured over 
the entire departure task. 

SVS Displays/Texture - SVS-only Analyses of variance were conducted on the main factors of Pilot, SVS 
Display Type (Size A, Size X, HUD) and Texture Type for the error performance measures with the 
following results: 

RMS lateral departure path error results (Figure 10) - SVS Display Type (F(2,279)=3.629, p<.028) was 
significant for the measure of RMS lateral path error as measured over the entire departure task. Post hoc 
tests (using SNK with a=.05), showed that with the HUD concept (mean=138 ft), significantly better tracking 
of the lateral path was obtained as compared to the Size A concept (mean=212 ft), but no appreciable 
differences were detectable with the Size X concept (mean=166 ft). Two distinct, overlapping subsets were 
formed: 1) HUD and Size X, and 2) Size X and Size A. In addition, there were no significant differences 
between Size A and Size X tracking performances. Although SVS Display Type was statistically significant 
for RMS lateral path error during the departure, it was not considered to be particularly meaningful 
operationally for an entire departure task. Performance differences varied directly with the display 
minification factor (at FOV= 60 degrees, minification factors were: A, 5.0; X, 2.6; HUD, 1.0), with the best 
performance occurring with the conformal HUD. Neither terrain Texture Type nor the interaction between 
Display Type and Texture Type were significant for this measure. 
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RMS speed-on-pitch error results -Display Type (F(2,279)= 187, p>.05), Texture Type (F(l,279)=.012, 
p>.05) and the interaction between Display Type and Texture Type (F(2,279)=.491, p>.05) were not 
significant for this measure over the entire departure task. 

Segment Analysis 

Baseline/S VS Displays - Segment analyses of variance were conducted on the main factors of Pilot and 
Display Type (baseline, Size A, Size X, HUD) for the individual segments of the approach and departure 
tasks for lateral and vertical path error and speed-on-pitch performance measures. Analysis of the data in 
segments 1, 2, 3 and segment 7 was not performed due to the small time spent in each segment 
(approximately 20 seconds or less for each segment). Note that the data in these excluded segments were 
included in the analysis presented above for the entire approach and departure. 


Approach Segment 4 - Straight and level at 8100 feet MSL 



□ Baseline 
■ Size A 

□ Size X 

□ HUD 


Figure 1 1 . Lateral RMS errors for all pilots by task segment. 

RMS lateral path error results (Figure 11) - Display Type (F(3,331)=57.656, p<.001) was highly significant 
for the measure of RMS lateral path error during segment 4. Post hoc tests (using SNK with a=.05) showed 
that the baseline concept (mean=149 ft) produced significantly poorer tracking of the lateral path as 
compared to the three SVS concepts: Size A (mean=56 ft), Size X (mean=49 ft) and HUD (mean=39ft). 
There were no significant differences among the SVS concepts for this measure, although differences were 
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found in the SVS Displays/Texture analysis below (the latter analysis has more statistical power to 
discriminate differences). 
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Figure 12. Vertical RMS error for all pilots. 


RMS vertical path error results (Figure 12) - Display Type (F(3,331)=.816, p<.486) was not significant for 
the measure of RMS vertical path error during segment 4. 

SVS Displays/Texture - SVS-only Analyses of variance were conducted on the main factors of Pilot, SVS 
Display Type and Texture Type for the lateral and vertical error performance measures with the following 
results: 


RMS lateral path error results (Figure 11) - SVS Display Type (F(2,279)=9.168,p<.001) was significant for 
the measure of RMS lateral path error during segment 4. Post hoc tests (using SNK with a=.05) showed that 
perfomance with the HUD (mean=39 ft) was statistically better than Size A (mean=56 ft) and Size X 
(mean=49 ft). The differences between Size A and Size X could not be discriminated. The improved 
performance was attributed to the HUD’s unity magnification factor, but the performance differences were 
not considered particularly meaningful operationally for this segment. Neither terrain Texture Type nor the 
interaction between Display Type and Texture Type were significant for this measure. 

RMS vertical path error results (Figure 12) -SVS Display Type (F(2,279)=5.683, p<.004) was significant for 
the measure of RMS vertical path error during segment 4. Post hoc tests (using SNK with a=.05) showed 
that with the Size A concept (mean=46 ft), significantly poorer tracking of the vertical path was obtained as 
compared to the other two SVS concepts: Size X (mean=39 ft) and HUD (mean=37 ft). The differences 
between Size X and the HUD could not be discriminated. The decreased performance was attributed to the 
increased minification factor of the Size A display, although the effect was not considered to be particularly 
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meaningful operationally for this segment. Neither terrain Texture Type nor the interaction between Display 
Type and Texture Type were significant for this measure. 

Approach Segment 5-3 degree descending turn to final 

RMS lateral path error results (Figure 11) - Display Type (F(3,331)=60.864, pc.OOl) was highly significant 
for the measure of RMS lateral path error during segment 5. Post hoc tests (using SNK with a=.05), showed 
that the baseline concept (mean=351 ft) produced significantly poorer tracking of the lateral path as compared 
to the three SVS concepts: Size A (mean=91 ft), Size X (mean=97 ft) and HUD (mean=68 ft). There were no 
significant differences among the SVS concepts for this measure, although differences were found in the SVS 
Displays/Texture analysis below (the latter analysis has more statistical power to discriminate differences). 

RMS vertical path error results (Figure 12) - Display Type (F(3,331)=13.741, p<.001) was highly significant 
for the measure of RMS vertical path error during segment 5. Post hoc tests (using SNK with a=.05), showed 
that the baseline concept (mean=105 ft) produced significantly poorer tracking of the vertical path as 
compared to the three SVS concepts: Size A (mean=47 ft), Size X (mean=40 ft) and HUD (mean=34 ft). 
There were no significant differences among the SVS concepts for this measure, although differences were 
found in the SVS Displays/Texture analysis below (the latter analysis has more statistical power to 
discriminate differences). 

SVS Displays/Texture - SVS-only Analyses of variance were conducted on the main factors of Pilot, SVS 
Display Type and Texture Type for the lateral and vertical error performance measures with the following 
results: 

RMS lateral path error results (Figure 11) - SVS Display Type (F(2,279)=10.105,p<.001) was significant for 
the measure of RMS lateral path error during segment 5. Post hoc tests (using SNK with a=.05) showed that 
perfomance with the HUD (mean=68 ft) was statistically better than Size A (mean=91 ft) and Size X 
(mean=97 ft). The differences between Size A and Size X could not be discriminated. The improved 
performance was attributed to the HUD’s unity magnification factor, but the performance differences were 
not considered particularly meaningful operationally for this segment because of the limited fidelity of the 
simulated HUD. Neither terrain Texture Type nor the interaction between Display Type and Texture Type 
were significant for this measure. 

RMS vertical path error results (Figure 12) -SVS Display Type (F(2,279)=4.143, p<.017) was significant for 
the measure of RMS vertical path error during segment 5. Post hoc tests (using SNK with a=.05) showed 
that the Size A concept (mean=47 ft) produced significantly poorer tracking of the vertical path as compared 
to the HUD (mean=34 ft) concept. The differences between Size X (mean=40 ft) and the HUD could not be 
discriminated as well as the differences between Size X and Size A. The decreased performance was 
attributed to the increased minification factor of the Size A display, although the effect was not considered to 
be particularly meaningful operationally for this segment because of the limited fidelity of the simulated 
HUD. Neither terrain Texture Type nor the interaction between Display Type and Texture Type were 
significant for this measure. 

Departure Segment 6 - 050 degree heading with speed-on-pitch 

RMS lateral path error results (Figure 11) - Display Type (F(3,315)=26.571, p<.001) was highly significant 
for the measure of RMS lateral path error during segment 6. Post hoc tests (using SNK with a=.05), showed 
that the baseline concept (mean=587 ft) produced significantly poorer tracking of the lateral path as compared 
to the three SVS Concepts: Size A (mean=265 ft), Size X (mean=218 ft) and HUD (mean=169 ft). There 
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were no significant differences among the SVS concepts for this measure, although differences were found in 
the SVS Displays/Texture analysis below (the latter analysis has more statistical power to discriminate 
differences). 

RMS speed-on-pitch error results (Figure 12) - Display Type (F(3,315)=.901, p>.05) was not significant for 
the measure of RMS speed-on-pitch error during segment 6. 

Other RMS error results - Within the SVS-only analyses, neither terrain Texture Type nor the interaction 
between Display Type and Texture Type were significant (p>.05) for either RMS lateral path error or RMS 
speed-on-pitch error during segment 6. However Display Type(F(2,267)=4.738, p<.01) was significant for 
RMS lateral path error. Post hoc tests (using SNK with a=.05) showed that the Size A concept (mean=265 ft) 
produced significantly poorer tracking of the lateral path as compared to the HUD concept(mean=169 ft). 
The differences between Size X (mean=218 ft) and the HUD could not be discriminated, nor could the 
differences between Size X and Size A. The decreased performance was attributed to the increased 
minification factor of the Size A display, although the effect was not considered to be particularly meaningful 
operationally for this segment because of the limited fidelity of the simulated HUD. 

Departure Segment 8 - 059 Radial from SXW VOR 

RMS lateral path error results (Figure 11) - Display Type (F(3,331)=22.612, p<.001) was highly significant 
for the measure of RMS lateral path error during segment 8. Post hoc tests (using SNK with a=.05), showed 
that the baseline concept (mean=504 ft) produced significantly poorer tracking of the lateral path as compared 
to the three SVS concepts: Size A (mean=143 ft), Size X (mean=108 ft) and HUD (mean=91 ft). There were 
no significant differences among the SVS concepts for this measure. 

RMS speed-on-pitch error results (Figure 12) - Display Type (F(3,315)=.958, p>.05) was not significant for 
the measure of RMS speed-on-pitch error during segment 8. 

Other RMS error results - Within the SVS-only analyses, neither Display Type, terrain Texture Type nor the 
interaction between Display Type and Texture Type were significant (p>.05) for either RMS lateral path error 
or RMS speed-on-pitch error during segment 8. 

Inferences from the Quantitative Results 

Approach Path 

In almost all of the approach analyses, significantly poorer tracking of both the lateral and vertical paths was 
obtained with the baseline concept as compared to the three SVS Concepts. While the guidance commands to 
correct for lateral and vertical path errors during approach were somewhat different, the magnitudes of the 
differences obtained may not be attributed to that factor alone. In the pre-experiment testing, approximate 
performance was obtained using both control laws. Improved tracking may be more attributed to the 
guidance symbology differences - specifically, to the tunnel or pathway and flight path marker symbology 
versus the conventional attitude flight director symbology. Note that the control law was the same for the 
departure, yet performance differences were obtained there as well. Similar results have been reported in 
numerous research studies confirming the advantages of making manual approaches using a tunnel because of 
the anticipatory nature of the display. Research has long established the benefits of prediction and preview 
(e.g., Lintern, Roscoe, & Sivier, 1990) and presentation of this information in a 3-D perspective (Haskell & 
Wickens, 1993; Wickens & Prevett, 1995; Theunissen, 1997). Therefore, the finding that the SVS guidance 
consisting of an integrated single cue flight guidance symbol (the ghost airplane on approach) with a tunnel 
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and velocity vector compared to a baseline concept with no tunnel was not surprising, but did confirm the 
desirability to include the use of a “pathway-in-the-sky” as an essential element in a synthetic vision system. 

Among the three SVS concepts, statistically significant tracking performance effects for both the lateral and 
vertical paths were not considered to be operationally meaningful. When they did occur, performance 
differences usually varied directly with the display minification factor, with the best performance occurring 
with the conformal HUD. Similar results have been reported in the previous SVS flight and simulation tests, 
and the current results verify the finding that with pilot selectable FOV, effective applications of SVS display 
technology can be accomplished in aircraft equipped with HDDs as small as Size-A (5.25” wide by 5” tall), 
thus confirming the potential of these SVS concepts as retrofit candidates for replacing current displays with 
synthetic vision technology. 

Additionally, no significant differences were found for Texture Type or the interaction between Display Type 
and Texture Type. These results also agree with the previous findings of SVS flight and simulation tests in 
which no objective measures revealed texture effects. 

Departure Path 

The lateral path tracking performance results for departure mirrored the approach path results. This finding 
was something of a surprise, in that, unlike the SVS concepts for approach, no tunnel was presented during 
the departure task for the SVS concepts. Also, the guidance commands used to drive the guidance 
symbologies were identical. Thus the only elements available to which the performance differences might be 
attributed include the dual cue, attitude -based symbology of the baseline concept versus the single cue, flight 
path marker-based symbology of the SVS concepts, and the fact that the baseline concept would require the 
pilot to redirect attention to the Navigation Display (TAWS, VSD) to acquire terrain information, while 
terrain information was present on the SVS tactical display. Either or both potential explanations imply a 
direct correspondence with workload results for the departure task (i.e., poorer tracking performance because 
of increased workload demands), which are presented in a later section. While most of the within SVS 
display concepts comparisons produced no statistically meaningful results, they, too, mirrored the approach 
path results in that the trends in performance differences always varied directly with the display minification 
factor, with the best performance occurring with the conformal HUD. 

Required Navigation Performance (RNP) 

RNP is a statement of the navigation performance accuracy necessary (see Appendix B) for operation within 
a defined airspace (RTCA DO-236A, 2000). RNP airspace is a generic term referring to airspace, routes, and 
legs, where minimum navigation performance requirements have been established and aircraft must meet or 
exceed that performance to fly in that airspace. RNP type is a designator according to navigational 
performance accuracy in the horizontal plane (lateral and longitudinal position fixing). This designator 
invokes all of the navigation performance requirements associated with the applicable RNP number, which is 
a containment value. For example, RNP-1 means that for at least 95% of the time the navigational 
performance in the horizontal plane, or the total horizontal system error, is less than 1.0 nmi. In addition to 
requiring 95% positioning accuracy for RNP operations, these types of procedures also require integrity of the 
positioning accuracy at 99.999% at 2 x RNP number. In our example above with an RNP-1, the position 
accuracy within 2.0 nmi of the ownship (2 x RNP value of 1 .0 nmi) would have to be guaranteed to be correct 
99.999% of the time to enable RNP-1 operations. Vertical navigation (VNAV) capability further enhances 
flight operations by enabling the specification of a flight path vertically for the lateral flight path. VNAV 
ensures that for at least 99.7% of the time the navigational performance in the vertical plane, or the total 
vertical system error, is less than a specified altitude deviation measure based on the airspace being flown in 
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(below 5000 feet MSL, 5000-10000 feet MSL, above 10000 feet MSL) and the type of flight operation (level 
flight/climb/descent or flight along specified vertical profile) being performed. For the approach-departure 
task in this experiment, the approach begins at 8200 feet MSL and is never lower than 5000 feet MSL. 
Therefore, since the pilot was flying a specified vertical profile, the required RNP vertical accuracy was 300 
feet. 


Horizontal RNP Summary for Display Type 
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Figure 13. Summary of the approach horizontal RNP for all display types averaged across all subjects. 

Figure 13 shows the horizontal FTE distribution for the display concepts using the bin widths of 0.05 nmi for 
the simulated EGE Visual 07 Approach. The path steering error component of the RNP calculation includes 
both FTE and display error. For this analysis, display error was assumed to be negligible; therefore FTE was 
the only component of path steering error. Also assumed for this analysis was that the other two components 
(path definition error and position estimation error) of the RNP calculation would be equivalent across the 
display concepts evaluated. The SVS concepts were able to achieve a horizontal FTE navigational accuracy 
of 0.05 nmi at least 95% of the time; while the baseline concept was able to achieve a horizontal FTE 
navigational accuracy of 0.15 nmi at least 95% of the time. As such, based on the FTE distributions shown, 
the SVS concepts (Size A, Size X, HUD) would enable horizontal RNP-type operations that were three times 
smaller than those that would be allowed with the baseline EADI concept. The complete RNP results can be 
found in Appendix C in Figures 46 through 49. 
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Vertical RNP Summary for Display Type 
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Figure 14. Summary of approach vertical RNP for all display types averaged across all subjects. 

Figure 14 shows the vertical FTE distribution for the display concepts using the bin widths of 50 feet. The 
vertical path steering error component of the vertical navigation (VNAV) performance calculation includes 
both FTE and display error. For this analysis, display error was assumed to be negligible therefore the FTE 
was the only component of vertical path steering error. Also assumed for this analysis was that the other 
three components (altimetry system error, vertical path definition error, and horizontal coupling error) of the 
VNAV performance calculation would be equivalent across the display concepts evaluated. All SVS 
concepts (Size A, Size X, and HUD) were able to achieve a vertical FTE navigational accuracy of 300 feet at 
least 99.7% of the time, while the baseline concept was unable to achieve this required RNP accuracy. As 
such, based on the FTE distributions shown, the SVS concepts would enhance flight operations by enabling 
the specification of a flight path vertically for the lateral flight path. 

Subjective Results 

Pilots were asked structured questions after each run. At the end of the day, a semi-structured interview was 
conducted along with a SA-SWORD questionnaire (Vidulich & Hughes, 1991). Qualitative results in the 
form of pilot comments are summarized below: 

Post Run Questionnaire Results 

After each individual data run, subjects were asked to rate their mental workload for the combined task (both 
approach and departure) using the Modified Cooper-Harper (MCH) Workload Scale (Boff et al., 1998). A 
low MCH Workload rating represented a low mental workload task. They were also asked to respond to six 
statements (See Appendix D). The responses to the statements are reported in the positive sense for ease in 
data interpretation. For example, one statement used during the experiment was “It was difficult to interpret 
the guidance cues”; however, in this paper, the results are reported as if the statement were phrased “It was 
easy to interpret the guidance cues.” Although MCH Workload ratings were solicitated, responses to the six 
statements were not sought after the rare-event runs because they were not relevant. 
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Modified Cooper-Harper Workload Scale Results for Nominal and Rare Event Runs 


MCH Workload Rating for Display Type 


U) 


<0 

~o 

<s 

2 3 

J- 

o 


X 

o 



Baseline Size A Size X 

Display Type 


HUD 


Figure 15. Subject Pilot average MCH Workload ratings for each display type 

MCH Workload results for nominal runs (Figure 15) Baseline/SVS Displays - Display Type 
(F(3,314)=l 10.77, p<.001) was highly significant for the pilots’ mental workload (MCH Workload) rating. 
Post hoc tests (using SNK with a=.05) showed that the baseline concept (rating=4.13) with TAWS and VSD 
enhanced ND increased the pilots’ mental workload rating as compared to the three SVS concepts: Size A 
(rating=2.20), Size X (rating=1.73) and HUD (rating=2.31). The Size X concept taxed the pilots’ mental 
workload less than the other two SVS concepts but this difference was not considered practically significant. 
Consistent with previous results within the SVS Project, the Size A display is considered less effective 
because of the increased miniflcation factors entailed by its small size, and the HUD is considered less 
effective because of its fixed field of regard. But the differences were not considered practically significant. 
In general, the baseline concept required a moderately high operator mental effort to attain adequate system 
performance, while the SVS concepts required a low operator mental effort and better performance was 
obtained. 

MCH Workload results for rare event CFIT runs (Figure 15) baseline/SVS Displays - Display Type (F(3, 
15)=10.286, p<.001) was highly significant for the MCH Workload ratings. Post Hoc tests (using SNK with 
a=.05) indicated two unique subsets: 1) Size X and HUD and 2) Size A and baseline. The HUD concept 
(mean rating=1.25) and Size X concept (mean rating=1.25) were judged to require less pilot workload 
(minimal to low operator mental effort) than the Size A concept (mean rating=2.75) and the baseline concept 
(mean rating=3.75). Although the Size A concept (low to acceptable operator mental effort) had a lower 
workload rating than the baseline concept, the difference was not statistically differentiable. The rating for 
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the baseline concept with TAWS and VSD enhanced ND (moderately high operator mental workload) 
indicated that the mental work level was not acceptable and should be reduced. 

MCH Workload results SVS Displays/Texture - The Display Type results from the SVS-only analysis were 
consistent with the above Baseline/SVS Displays analysis. Terrain Texture Type (F(l,264)=6.602, p<.011) 
was statistically significant (but was not considered to be operationally significant) for the pilot’s mental 
workload rating. The mean rating was 1.98 for the photo-realistic texturing and 2.18 for the generic 
texturing, both equating to a low operator mental workload requirement. The interaction between SVS 
Display Type and Texture Type was not significant for this measure. 

Post Run Statement Responses for Nominal Runs 

Subjects were asked to rate each statement with a value between 1 and 6, where a 1 indicated a strong 
disagreement with the statement and a 6 represented strong agreement. Ratings of 2 and 5 were moderate 
disagreement or agreement, respectively, and ratings of 3 and 4 were slightly disagree or agree, respectively. 


IT WAS EASY TO DETERMINE AIRCRAFT POSITION WITH 
RESPECT TO TERRAIN 

(1=Strongly disagree; 6=Strongly agree) 



Display Type 


Figure 16. Average results of subjects’ responses to run statement 1. 

Run Statement 1 (Figure 16) - It was easy to determine aircraft position with respect to the terrain : Display 
Type (F(3,314)=l 11.140, p<.001) was highly significant for the measure of ease of determining aircraft 
position with respect to terrain. Post hoc tests (using SNK with a=.05) showed that it was judged harder to 
determine aircraft position with respect to terrain with the baseline concept (rating=3.48) with TAWS and 
VSD enhanced ND than it was with the three SVS concepts: Size A (rating=5.35), Size X (rating=5.57) and 
HUD (rating=5.33). There were no appreciable differences between the SVS concepts (even within the more 
powerful SVS-only ANOVA). 
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I WAS CONFIDENT IN THE TERRAIN INFORMATION 
CONVEYED BY THE DISPLAY 

(1=Strongly disagree; 6=Strongly agree) 



Display Type 


Figure 17. Average results of subjects’ responses to run statement 2. 

Run Statement 2 (Figure 17) - I was confident in my knowledge of separation from the terrain : Display Type 
(F(3, 3 14)=79.454, p<.001) was highly significant for the measure of a pilot’s confidence in his knowledge of 
the aircraft’s separation from the terrain. Post hoc tests (using SNK with a=.05) showed that the pilots were 
less confident in the knowledge of terrain clearance with the baseline concept (rating=3.69) with TAWS and 
VSD enhanced ND than with the three SVS concepts: Size A (rating=5.21), Size X (rating=5.38) and HUD 
(rating=5.26). There were no appreciable differences between the SVS concepts (even within the more 
powerful SVS-only ANOVA). 


IT WAS EASY TO INTERPRET THE GUIDANCE CUES 

(1=Strongly disagree; 6=Strongly agree) 



Display Type 


Figure 18. Average results of subjects’ responses to run statement 3. 

Run Statement 3 (Figure 18 ) - It was easy to interpret the guidance cues : Display Type (F(3,314)=50.372, 
p<.001) was highly significant for the measure of ease of interpreting guidance cues. Post hoc tests (using 
SNK with a = .05) showed that it was judged slightly harder to interpret the guidance cues with the baseline 
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display concept (dual cue guidance symbol, rating=4.23) than it was with the three SVS display concepts 
(single cue ball guidance symbol), which were indistinguishable (even within the more powerful SVS-only 
ANOVA): Size A (rating=5.29), Size X (rating=5.62) and HUD (rating=5.18). 


IT WAS EASY TO FOLLOW THE GUIDANCE CUES 

(1=Strongly disagree; 6=Strongly agree) 



Display Type 


Figure 19. Average results of subjects’ responses to run statement 4. 

Run Statement 4 (Figure 19) - It was easy to follow the guidance cues : Display Type (F(3,314)=33.717, 
p<.001) was highly significant for the measure of ease of following the guidance cues. Post hoc tests (using 
SNK with a = .05) showed that it was judged slightly harder to follow the guidance cues with the baseline 
display concept (dual cue guidance symbol, rating-4.00) than it was with the three SVS display concepts 
(single cue ball guidance symbol), which were indistinguishable (even within the more powerful SVS-only 
ANOVA): Size A (rating-4.95), Size X (rating-5.32) and HUD (rating-4.77). 


THE AMOUNT AND DENSITY OF DISPLAY INFORMATION 
WAS APPROPRIATE FOR THE TASK 

(1=Strongly disagree; 6=Strongly agree) 



Display Type 


Figure 20. Average results of subjects’ responses to run statement 5. 
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Run Statement 5 (Figure 20) - The amount and density of display information was appropriate to the task : 
Subject pilots were briefed during the training session that this statement referred to amount of clutter on the 
display. Display Type (F(3,314)=10.907, pc.OOl) was highly significant for the measure of interpretability of 
display information. Post hoc tests (using SNK with a = .05) showed that it was judged slightly harder to 
interpret the display information with the baseline display concept (rating=4.31) than it was with the three 
SVS display concepts, which were indistinguishable (even within the more powerful SVS-only ANOVA): 
Size A (rating=4.72), Size X (rating=5.07) and HUD (rating=4.69). 


I COULD PERFORM THIS TASK WITH EASE AND PRECISION 

(1=Strongly disagree; 6=Strongly agree) 



Display Type 


Figure 21. Average results of subjects’ responses to mn statement 6. 

Run Statement 6 (Figure 21) - I could perform the task with ease and precision : Display Type 
(F(3,314)=19.842, p<.001) was highly significant for the measure of performing the flying task with ease and 
precision. Post hoc tests (using SNK with a = .05) showed that it was judged slightly harder to perform the 
flying task with the baseline display concept (rating=4.02) than it was with the three SVS display concepts, 
which were indistinguishable (even within the more powerful SVS-only ANOVA): Size A (rating=4.85), Size 
X (rating=5.07) and HUD (rating=4.71). 

S A-S WORD Questionnaire Results 

The SA-SWORD (Vidulich & Hughes, 1991) for this experiment was administered to allow a statistical 
analysis of the pilot’s subjective assessment of the SA for each of the seven Display Configurations (baseline 
and photo-realistic and generic texturing on Size A, Size X and HUD). The definition of SA was defined as: 
The pilot’s awareness and understanding of all factors that will contribute to the safe flying of their aircraft 
under normal and non-normal conditions. 

Display Configuration was highly significant for the SA-SWORD ratings (F(6,90)=14.443, pO.OOl). Post 
hoc tests (using SNK with a = .05) showed that the photo-textured Size X display was judged to provide 
significantly better SA than all other display concepts tested. The photo-textured HUD was judged to provide 
significantly better SA than both Size A (generic and photo) concepts and the baseline concept but had no 
appreciable statistical differences from the generic-textured Size X or generic-textured HUD (Figure 22). 


38 


Higher SA ► Lower SA 

y 

Size X Photo HUD Photo Size X Generic HUD Generic Size A Photo Size A Generic Baseline 


♦ ♦ 

Any two means not underscored by the same two lines are significantly different. 

Figure 22. Comparative situational awareness among display concepts. 

Among the three types of SVS concepts (Size A, Size X, HUD), statistically the photo-realistic database was 
judged to afford significantly more SA than generic texturing for the Size-X display concept only. All other 
texturing effects within a given size were not discriminable. 

Pilot Workload Rating Results 

The subjects were asked to rate the workload using each display concept (baseline, Size A, Size X and HUD) 
for both the approach and departure separately. Workload was defined as: “the degree of cognitive 
processing capacity required to perform the flight task approach adequately”. Figure 23 shows the scale that 
the pilots used for the workload and SA ratings. 





Figure 23. Workload and SA rating scale. 
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Subject Workload Rating 
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Figure 24. Subjects’ rating of their workload for the approach and departure tasks. 


Approach Path 

Baseline/SVS Displays (Figure 24) - Display Type (F(3,93)=67.310, p<.001) was highly significant for the 
measure of pilot workload rating during the approach. Post hoc tests (using SNK with a = .05), showed that 
the baseline concept (rating=4.13) increased the pilots’ workload rating as compared to the three SVS 
concepts: Size A (rating=6.94), Size X (rating=8.16) and HUD (rating=7.19). The Size X concept taxed the 
pilots’ workload less than the other two SVS concepts. Consistent with previous results, the Size A display is 
considered less effective because of the increased minification factors entailed by its small size, and the HUD 
is considered less effective because of its fixed field of regard. In general, the baseline concept workload 
rating was somewhat high and the pilots’ workload rating with the SVS concepts was somewhat low during 
the approach. 

SVS displays/texture - The Display Type results from the SVS-only analysis were consistent with the above 
Baseline/SVS Displays analysis. Neither terrain Texturing Type or the interaction between SVS Display Type 
and Texture Type was significant for this measure during the approach. 

Departure Path 

Baseline/SVS Displays (Figure 24) - Display Type (F(3,93)=93.651, p<.001) was highly significant for the 
measure of pilot workload rating during the departure. Post hoc tests (using SNK with a = .05) showed that 
the baseline concept (rating=3.62) increased the pilots’ workload rating as compared to the three SVS 
Concepts: Size A (rating=6.87), Size X (rating=8.09) and HUD (rating=7.13). The Size X concept taxed the 
pilots’ workload less than the other two SVS concepts. Consistent with previous results, the Size A display is 
considered less effective because of the increased minification factors entailed by its small size, and the HUD 
is considered less effective because of its fixed field of regard. In general, the baseline concept workload 
rating was somewhat high and the pilots’ workload rating with the SVS concepts was somewhat low during 
the departure. 
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SVS displays/texture - The Display Type results from the SVS-only analysis were consistent with the above 
Baseline/SVS Displays analysis. Neither terrain Texturing Type or the interaction between SVS Display 
Type and Texture Type was significant for this measure during the departure. 

Pilot SA Rating Results 

Subjects were asked to rate their level of SA experienced during both the approach and departure task 
separately, for each display concept (See Figure 23 for rating scale). Note that the SA ratings were reversed 
scored (ie, 9 was very high SA and 1 was very low SA). 


Subject SA Rating 
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Figure 25. Subjects’ rating of SA for the approach and departure task. 


Approach Path 

Baseline/SVS Displays (Figure 25) - Display Type (F(3,93)=l 54.272, p<.001) was highly significant for the 
pilot’s SA rating during the approach. Post hoc tests (using SNK with a = .05), showed that the baseline 
concept (rating=3.94) was judged to have less situation awareness as compared to the three SVS concepts: 
Size A (rating=7.06), Size X (rating=8.72) and HUD (rating=7.19). The Size X concept was judged to 
provide better SA than the other two SVS concepts. Consistent with previous results, the Size A display is 
considered less effective because of the increased minification factors entailed by its small size, and the HUD 
is considered less effective because of its fixed field of regard. In general, the pilots’ SA with the baseline 
concept was somewhat low and their SA with the SVS concepts was high during the approach. 
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Texture Effects 
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Size X HUD 
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Display Type 
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■ photo-realistic 


Figure 26. Subjects’ rating of SA for Texture Type for the approach and departure task. 

SVS displays/texture - The Display Type results from the SVS-only analysis were consistent with the above 
Baseline/SVS Displays analysis. Terrain Texture (F(l,75)=l 1.239, p<.001) and the interaction between SVS 
Display Type and Texture Type (F(2,75)=4.664, p<.012) were significant for this measure during approach. 
The photo-realistic texture (mean = 7.88) obtained a higher mean situation awareness rating for the approach 
task than did the generic texture (mean = 7.44), but the significant interaction between SVS Display Type and 
Texture Type (shown in Figure 26) indicated that this effect was only present for the HUD (see Figure 26). 
The Texture Type effect was negligible for the HDDs. Operationally, these results were considered not 
significant, as the situation awareness ratings (varying from 6.6 to 8.9) were all high. 

Departure Path 

Baseline/SVS Displays (Figure 25) - Display Type (F(3,93)=283.305, p<.001) was highly significant for the 
pilot’s SA rating during the departure. Post hoc tests (using SNK with a = .05), showed that the baseline 
concept (rating=2.50) was judged to have considerably less situation awareness as compared to the three SVS 
concepts: Size A (rating=7.06), Size X (rating=8.75) and HUD (rating=7.09). The Size X concept yielded 
better pilot SA than the other two SVS concepts. Consistent with previous results, the Size A display is 
considered less effective because of the increased minification factors entailed by its small size, and the HUD 
is considered less effective because of its fixed field of regard. In general, the pilots’ SA rating with the 
baseline concept was low and their SA rating with the SVS concepts was high during the approach. 

SVS displays/texture - The Display Type results from the SVS-only analysis were consistent with the above 
Baseline/SVS Displays analysis. Terrain Texture (F(l,75)=l 1 .800, p<.001) and the interaction between SVS 
Display Type and Texture Type (F(2,75)=3.933, p<.024) were significant for this measure during departure. 
The photo-realistic texture (mean = 7.85) obtained a higher mean situation awareness rating for the departure 
task than did the generic texture (mean = 7.42), but the significant interaction between SVS Display Type and 
Texture Type (shown in Figure 26) indicated that this effect was only present for the HUD (see Figure 26). 
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The Texture Type effect was negligible for the HDDs. Operationally, these results were considered not 
significant, as the situation awareness ratings were all high. 

VSD Enhancement Questionnaire Results 

Subjects were asked to rate how much their SA was increased with the VSD during the approach and 
departure tasks using the: 1) baseline EADI with TAWS and VSD enhanced ND; 2) SVS Size A; 3) SVS 
Size X; and 4) SVS HUD. Subjects were asked to rate the VSD addition to the ND on a scale of 0 to 10 (0 
being 0% enhancement and 10 being 100% enhancement). 


Percent enchancement of SA by having VSD 



□ approach 
■ departure 


Display Type 


Figure 27. Percent enhancement of SA by having VSD for each display type. 


Approach Path 

During approach (Figure 27), Display Type (F(3, 63)=12.201, p<.001) was highly significant for the measure 
of the VSD percentage enhancement to SA. Post Hoc tests (using SNK with a=.05) indicated that the VSD 
was judged to have significantly enhanced SA for the baseline concept (mean=61 percent enhancement) with 
TAWS and VSD enhanced ND as compared to the SVS concepts (mean=34 percent for Size A, mean=22 
percent for Size X and mean=38 percent for HUD). Also, the SNK test showed that the Size X rating was 
significantly different from that of the HUD but had no appreciable difference from that of the Size A rating. 
In addition, there were no significant differences between the ratings for the Size A and the HUD. These 
within-SVS concept results are consistent with the previously found subjective preferences for the Size X 
display concept (interpreting the results as meaning that the Size X display required less terrain 
enhancement). 

Departure Path 

During the departure (Figure 27), Display Type (F(3, 63)= 1 1.125, p<.001) was highly significant for the 
measure of the VSD percentage enhancement to SA. Post Hoc tests (using SNK with a=.05) indicated that 
the VSD was judged to have significantly enhanced SA for the baseline concept (mean =59 percent 
enhancement) with TAWS and VSD enhanced ND as compared to the SVS concepts (mean=36 percent for 
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Size A, mean=23 percent for Size X and mean=37 percent for HUD). The SVS concept rating differences 
were not statistically detectable. 

Inferences from the On-site Qualitative Results 

In all analyses of ratings related to workload and situation awareness, pilots judged the baseline concept to 
have higher workload and lower situation awareness as compared to the three SVS Concepts. Among the 
SVS concepts, rating differences were small, but the Size X display garnered the better ratings, followed by 
the HUD and the Size A display. These results are entirely consistent with the numerous research studies 
previously discussed. The Size A display is considered less effective because of the increased minification 
factors entailed by its small size, and the HUD is considered less effective because of its fixed field of regard. 
Not surprisingly, the VSD was judged to improve terrain awareness most for the baseline concept, where the 
only other terrain information was present on the TAWS. Among the SVS concepts, the ratings varied in a 
manner reflective of the workload and SA results above. 

Of particular interest in light of the objective data results concerning the guidance commands and symbology 
differences during approach and departure are the associated subjective results. On approach, the tunnel or 
pathway and flight path marker symbology were judged easier to use and interpret than the conventional 
attitude flight director. On departure, the single cue ball referenced to the flight path marker garnered better 
ratings than the dual cue flight director bars referenced to the attitude symbol. These subjective results for 
departure, along with the findings that the baseline concept increased the pilots’ workload rating as compared 
to the three SVS Concepts, are in direct correlation with the potential explanations of lateral path tracking 
performance results for departure advanced earlier in the “Inferences from the Quantitative Data” section (i.e., 
poorer tracking performance with the baseline concept because of increased workload demands imposed by 
the guidance symbology and the requirement to scan for terrain information). 

Post Experiment Questionnaire 

At the conclusion of the experiment, each subject was given a “take home” questionnaire packet to be 
completed and mailed back. Fourteen of the sixteen questionnaires were returned and the responses are 
included in Appendix G. The purpose of the questionnaire was to provide feedback for improving SVS 
displays. In addition, the pilot subjects were asked for their opinion on possible operational benefits of SVS. 
A summary of the subjects’ responses are provided below: 

1 . The Size X with photo-realistic terrain was the preferred display. Though photo-realistic texturing was 
preferred, subjects rated the generic texture as acceptable. The larger Size X provided more SA compared to 
the Size A display. The Size A was adequate to perform the task; however, its small size made symbology 
interpretation difficult. Color terrain was preferred over the monochrome terrain due to its intuitive view of the 
outside world. Also, color symbology made interpretation of the Size X display easier than the monochrome 
HUD. 

2. Subjects felt the HUD with a terrain database image enhanced SA. For crew coordination purposes, subjects 
recommended both the pilot and co-pilot have an SVS enhanced HUD if it were to be implemented in the 
commercial fleet without SVS head down displays. The pilots recommended that the presentation of synthetic 
terrain on the HUD be pilot selectable. They were concerned that the synthetic terrain would clutter the HUD 
in VMC or when breaking out of a cloud bank. 

3. Overall, pilots liked the tunnel and ghost airplane guidance concept on the SVS displays. The pilots liked the 
anticipatory cues the tunnel gave for path changes. The ghost airplane provided a target for the velocity vector, 
making it easy to fly the path. 

4. Pilots preferred having selectable FOV. A 30 degree FOV was preferred for approach and 60 degree FOV for 
the departure. The smaller FOV allowed pilots to “fine tune” the approach on final. The larger FOV was used 
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for departure mainly because of the large angle of attack imposed, which required more vertical SA. In 
addition, 60 degree FOV provided more horizontal terrain awareness for the departure maneuver. 

5. Subjects recommended both a HUD and HDD SVS solution for current and future aircraft. 

6. Pilots felt that SVS would best support arrival and approach operations, although they felt SVS would be 
beneficial in all phases of flight including taxi operations. 

Summary of Inferences 

SVS concepts are dramatically different from the displays of today’s commercial cockpit, yet with little 
training the subjects were able to fly the complex task with better performance compared to a typical EFIS 
display, with which they were very familiar. In addition, the SVS reduced mental workload and provided 
better SA for terrain awareness. Combining this precision guidance capability with the intuitive nature of the 
SVS terrain information has the potential to eliminate precursors to accidents that have low visibility as a 
causal factor. The results of the present experiment confirm the efficacy of SVS to enhance a flight crew’s 
terrain awareness and be proactive to avoid CFIT accidents. Although TAWS and VSD provide a graphical 
presentation of terrain information, these displays require the flight crew to integrate the information from the 
two separate 2-D displays in order to form a mental image of the 3 -dimensional terrain. Because SVS 
provides a dynamic and accurate display of the outside world, synthetic vision allows the flight crew to 
cognitively map the terrain information to an analog of what it naturally represents; a mountain in the 
synthetic vision display looks like a mountain in the real world. Thus, interpretation of these displays is 
intuitive and allows the flight crew to identify emergent situations that can cause CFIT accidents. 

The scenario used in the present experiment represents such an example. In this case, the pilot flew multiple 
approaches before being exposed to the CFIT scenario. Often, these accidents happen with an otherwise 
experienced flight crew who had flown the same route many times previous to the CFIT encounter. Rather 
than one major error on the part of the flight crew, CFIT situations often arise from a series of small 
seemingly inconsequential events that collectively result in an accident. Numerous examples of such 
accidents illustrate this point (e.g., Cali, 1995; Khatmandu, 1990; Madrid, 1983), and the CFIT scenario used 
in the experiment was no different than conditions that caused these accidents. Instead, many of the pilots 
that experienced a CFIT noted that they were lulled into a state of complacency. They had flown the same 
departure twenty-one times previously and each time they noted that they were getting TAWS indications that 
high terrain was present, but also knew that the safe passage was available through the “notch” in the 
mountain. They never expected that something may go wrong with their flight guidance and, therefore, 
“trusted” that they were on-path despite indications on the VSD that something was wrong. Essentially, they 
lost their vigilance. A similar situation happened to the pilots who flew the SVS, but they had one advantage 
— synthetic vision. They, too, reported that the last departure did not initially appear to be any different than 
the previous twenty-one missed approaches. However, there was one key difference. The velocity vector 
seemed to be leading them directly into the mountain rather than through the path between the peaks. 
Therefore, this allowed for “recognition primed” decision making to know that something was “not right” and 
that they needed to crosscheck other instruments. In fact, even before TAWS would have alerted the flight 
crew, several pilots began to change their flight path and began to laterally maneuver the aircraft so they 
could continue the approach through the mountain pass. With the baseline displays, there would be no way 
that a flight crew could have executed such a maneuver because of the limited resolution of the terrain 
presentation. Rather, upon hearing the TAWS alert, the flight crew is left with the singular solution of “pull 
up, pull up” which because of reduce climb performance would not have allowed them to escape colliding 
with the terrain. Although new technologies are being developed to allow TAWS to provide both lateral and 
vertical escape guidance, it still cannot equate to the situation awareness provided by having a realistic 
“picture” of the outside world available to the flight crew at all times. As one of the evaluation pilots 
observed, “a picture may be worth a thousand words, but with synthetic vision, a picture is worth a thousand 
lives.” 


45 



The quantitative analyses showed the subject pilots flew the SVS displays with greater precision compared to 
the baseline concept. These results were obtained for the approach task for both lateral and vertical error over 
the entire approach and within each individual segment, with the single exception of vertical error for 
segment 4. For that segment, which involved flying a straight and level path with no changes, no vertical path 
differences were detectable. Similar results were obtained for the lateral error for the departure task, where 
the control law guidance provided to the pilot was the same between the baseline and SVS display concepts. 
The major difference on departure between the baseline concept and its SVS Size A display counterpart was 
the terrain presentation and the velocity vector. No pathway was present. And yet the lateral FTE was 
significantly lower for the SVS displays. Differences in the RMS speed-on-pitch error were not significant 
throughout the departure task. 

Thus for both the approach and departure tasks, the flight guidance symbologies embodied within the SVS 
displays provided superior performance compared to the flight guidance symbologies embodied within the 
baseline concept. Also, the run questionnaires support this conclusion, as pilots rated the baseline flight 
guidance as requiring high mental workload and as being more difficult to follow than the SVS display 
guidance. From these data, the conclusion is the presentation of the guidance symbology is highly beneficial, 
enhancing the pilot’s ability to perform the task. Though the tunnel and ghost presentation of approach 
guidance used very different guidance laws from the departure guidance, the subject task is the same as the 
single integrated guidance symbol (the “ball”) used for departure. The task for both of these guidance 
concepts is to simply put the velocity vector on the guidance cue while the dual cue flight director requires 
zeroing two separate guidance symbols. Subjective comments showed pilots liked the “look ahead” features 
that the tunnel and ghost airplane provided over the ball. The ghost airplane gave subjects an indication of 
where the guidance was going. Additionally, the tunnel pathway allowed pilots to anticipate on-coming turns 
in the path. 

Among the SVS concepts, objective performance differences and subjective rating differences were small. 
When such differences existed in FTE, the trends always varied directly with the display minification factor, 
with the best performance occurring with the conformal HUD. When such differences existed in subjective 
rating differences, the Size X display garnered the better ratings, followed by the HUD and the Size A 
display. The Size A display is considered less effective because of the increased minification factors entailed 
by its small size, and the HUD is considered less effective because of its fixed field of regard and its 
monochrome presentation. Both objective and subjective results are entirely consistent with the numerous 
research studies conducted within the SVS Project. 


Conclusions 

The rare event portion of the simulation experiment demonstrated quite dramatically that a synthetic vision 
system (SVS) will improve the pilot’s ability to detect and avoid a potential Controlled Flight Into Terrain 
(CFIT) event compared to the baseline 757 EFIS display system. This CFIT benefit has been advanced as 
one primary motivation for SVS displays (another primary motivation is to replicate the operational benefits 
of flight operations in bright, clear, sunny day conditions, regardless of the outside weather). In addition, the 
backdrop retrofit display simulation experiment confirmed once again the prior flight test and simulator 
results revealing the enhanced situation awareness, reduced workload, and reduced flight technical error 
provided by all of the SVS (HDD and HUD) concepts compared to the baseline display configuration, 
regardless of display size. These additional results firmly establish the SVS HDD retrofit concept approach 
as viable for operations in a realistic, terrain-challenged environment. 
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Appendix A: SYS Guidance 


The guidance presented to the pilot depended on the task (approach or departure) and display type (baseline 
or SVS). Table 6 indicates the guidance symbology presented to the pilot. For baseline data runs, the flight 
guidance was a typical dual cue (Figure 28) flight director providing guidance to the approach and departure 
path. However, the vertical guidance for the departure path was speed on pitch. For the SVS data runs, pilots 
flew the approach procedure with tunnel (or highway-in-the-sky) and ghost airplane guidance (Figure 29, 
Merrick & Jeske, 1995). The ghost airplane, rendered as two triangles and an aiming circle, is pursuit type 
guidance which is 30 seconds ahead on the intended path. The ghost airplane would pitch and yaw giving the 
pilot the intended movement of the guidance. The departure guidance symbology for the SVS runs was an 
integrated single cue symbol (or “the ball”) providing the same lateral and vertical path guidance as the 
baseline display concept. The guidance laws for the flight directors are shown in Figures 30 - 32. In 
addition, DME on the 059 radial of SXW was displayed for the departure. The ghost airplane pursuit 
guidance is described in the Merrick reference. 


Table 6. Flight guidance symbology presented to pilot based on phase of flight and display type 



Baseline display 

SVS display 

Approach 

Dual cue (pitch & roll bars) 

Ghost airplane and tunnel 

Departure 

Dual cue (pitch & roll bars) 

Integrated ball (speed on pitch/lateral path) 



Figure 28. Baseline approach and departure dual cue (pitch and roll command bars) flight director symbology. 
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Figure 29. SVS guidance symbology: approach (left) ghost airplane with tunnel and departure (right) integrated single 

cue guidance (“ball”). 
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Figure 30. Lateral flight director for baseline concept approach tasks and for baseline and SVS concepts departure tasks. 
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Figure 32. Speed-on-pitch flight director for all display concepts departure tasks. 


51 










Appendix B: Required Navigation Performance 


Required navigation performance (RNP) is a statement of the navigation performance accuracy necessary for 
operation within a defined airspace. RNP airspace is a generic term referring to airspace, routes, and legs, 
where minimum navigation performance requirements have been established and aircraft must meet or 
exceed that performance to fly in that airspace. The system performance requirements for RNP Area 
Navigation (RNAV) is that each aircraft operating in RNP airspace shall have total system error components 
in the cross-track and along-track directions that are less than the RNP value 95% of the flying time. RNP 
type is a designator according to navigational performance accuracy in the horizontal plane (lateral and 
longitudinal position fixing). This designator invokes all of the navigation performance requirements 
associated with the applicable RNP number, which is a containment value. For example, RNP-1 means that 
for at least 95% of the time the navigational performance in the horizontal plane, or the total horizontal 
system error, is less than 1.0 nautical mile (nmi). In addition to requiring 95% positioning accuracy for RNP 
operations, these types of procedures also require integrity of the positioning accuracy at 99.999% at 2 x RNP 
number. In our example above with an RNP-1, the position accuracy within 2.0 nmi of the ownship (2 x RNP 
value of 1.0 nmi) would have to be guaranteed to be correct 99.999% of the time to enable RNP-1 operations. 

There are three lateral components of navigation error: path definition error, path steering error, and position 
estimation error (RTCA, 2000). These errors, defined in the following, represent the total horizontal system 
error of the airplane and are the difference between the aircraft’s true position and desired position (Figure 
33): 


The path definition error is the difference between the defined path and the desired path at a specific 
point. 

The path steering error is the distance from the estimated position to the defined path. It includes 
both the flight technical error (FTE) and display error. FTE is the accuracy with which the aircraft is 
controlled as measured by the indicated aircraft position with respect to the indicated command or 
desired position. 

The position estimation error, also referred to as the ship’s actual navigation performance (ANP), is 
the difference between the true position and the estimated position. 
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Figure 33. Lateral components of navigation error terms 

Vertical navigation (VNAV) capability further enhances flight operations by enabling the specification of a 
flight path vertically for the lateral flight path. The system performance requirements for VNAV are that for 
at least 99.7% of the time the navigational performance in the vertical plane, or the total vertical system error, 
is less than a specified altitude deviation measure based on the airspace being flown in (below 5000 feet 
MSL, 5000-10000 feet MSL, above 10000 feet MSL) and the type of flight operation (level 
flight/climb/descent or flight along specified vertical profile) being performed (Table 7). 

There are four vertical components of navigation error: altimetry system error, vertical path steering error, 
vertical path definition error, and horizontal coupling error (RTCA, 2000). These errors, defined in the 
following, represent the total vertical system error of the airplane and are the difference between the aircraft’s 
true vertical position and desired vertical position at the true lateral position (Figure 34): 

Altimetry system error is the error attributable to the aircraft altimetry installation, including position 
effects resulting from normal aircraft flight attitudes. 

The vertical path steering error is the distance from the estimated vertical position to the defined 
path. It includes both FTE and display error. 

The vertical path definition error is the vertical difference between the defined path and the desired 
path at the estimated lateral position. 

The horizontal coupling error is the vertical error resulting from horizontal along track position 
estimation error coupling through the desired path. 
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Table 7. Vertical Accuracy Performance Requirements 


Error Source 

Level Flight Segments 
and Climb/Descent 
Intercept of Clearance 
Altitudes (MSL) 

Approach 
along Specified 
Vertical Profile 
(MSL) 


At or Below 
5000 ft 

5000 ft to 
10000 ft 

Above 
10000 ft 

At or Below 
5000 ft 

5000 ft to 
10000 ft 

Above 10000 ft 

Altimetry 

90 ft 

200 ft 

250 ft 

140 ft 

265 ft 

350 ft 

RNAV 

Equipment 

50 ft 

50 ft 

50 ft 

100 ft 

150 ft 

220 ft 

Flight 

Technical 

150 ft 

240 ft 

240 ft 

200 ft 

300 ft 

300 ft 

Total Root- 
Sum-Square 
(RSS) 

190 ft 

320 ft 

350 ft 

265 ft 

430 ft 

510 ft 


True Position 



Defined Path 


Desired Path 
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Appendix C: Miscellaneous Figures 



Figure 35. Baseline display. 
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Figure 36. HUD with generic textured terrain. 



Figure 37. HUD with photo-realistic textured terrain. 





Figure 38. Size A with generic textured terrain. 



Figure 39. Size A with photo-realistic textured terrain. 
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Figure 40. Size X with generic textured terrain. 



Figure 41 . Size X with photo- realistic textured terrain. 
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Figure 42. Baseline display on departure. 



Figure 43. Size X display on departure. 
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Figure 44. Size X photo-textured terrain on final to EGE runway 7. 



Figure 45. Size X photo-realistic terrain in a left turn. 
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Figure 46. Lateral and Vertical FTE for the baseline display. 
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Figure 47. Lateral and Vertical FTE for the SVS Size A display. 
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Figure 48. Lateral and Vertical FTE for the SVS Size X display. 
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Figure 49. Lateral and Vertical FTE for the SVS HUD. 


64 


Appendix D: Between Run Questionnaires 



Difficulty Level Operator Demand Level Rating 


VERY EASY, 
Highly desirable 

Operator mental effort is minimal and 
desired performance is easily attainable 

1 

EASY, 

Desirable 

Operator mental effort is low and 
desired performance is attainable 

2 

FAIR, 

Mild Difficulty 

Acceptable operator mental effort is required 
to attain adequate system performance 

3 


MINOR BUT ANNOYING 
DIFFICULTY 

Moderately high operator mental effort is required 
To attain adequate system performance 

4 

MODERATELY OBJECTIONABLE 
DIFFICULTY 

High operator mental effort is required 
To attain adequate system performance 

5 

VERY OBJECTIONABLE BUT 
TOLERABLE DIFFICULTY 

Maximum operator mental effort is required 
To attain adequate system performance 

6 


MAJOR DIFFICULTY 

Maximum operator mental effort is required 
To bring errors to moderate level 

7 

MAJOR DIFFICULTY 

Maximum operator mental effort is required 
To avoid large or numerous errors 

8 

MAJOR DIFFICULTY 

Intense operator mental effort is required 
To accomplish task, but frequent or 
Numerous errors persist 

9 


IMPOSSIBLE 


Instructed task cannot be accomplished reliably 


10 


Figure 50. Modified Cooper Harper scale asked between runs. 
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RUN QUESTIONS RE 


STRONGLY 

MODERATELY 

SLIGHTLY 

SLIGHTLY 

MODERATELY 

STRONGLY 

DISAGREE 

DISAGREE 

DISAGREE 

AGREE 

AGREE 

AGREE 

1 

2 

3 

4 

5 

6 


1 . ) IT WAS EASY TO DETERMINE AIRCRAFT POSITION 

WITH RESPECT TO THE TERRAIN: 

COMMENTS: 

2. ) I WAS CONFIDENT IN MY KNOWLEDGE OF SEPARATION FROM 

TERRAIN: 

COMMENTS: 

3. ) IT WAS EASY TO INTERPRET THE GUIDANCE CUES 

COMMENTS: 

4. ) IT WAS EASY TO FOLLOW THE GUIDANCE CUES: 

COMMENTS: 

5. ) DISPLAY INFORMATION WAS CONFUSING AT TIMES: 

COMMENTS: 

6. ) I COULD PERFORM THIS TASK WITH EASE AND PRECISION: 


COMMENTS: 


Figure 5 1 . Run Questionnaire asked between each data run. 
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Appendix E: Miscellaneous Data Tables 



Table 9. Subject Experience 

Subject Years flying Total hours 



Average 19.625 8175.875 

Min 11 1800 

Max 33 15500 
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Appendix F: Pilot Responses to Semi Structured Interview 


1 . Please rate the ease of performing the approach to rwy 7 using the traditional EADI with flight director and 
TAWS + VSD 

SUBJECT 1: Rating of 5 Familiarity of approach and use of displays helped, but the displays lacked SA and 
was difficult interpreting the flight guidance 

SUBJECT 2: Rating of 4. Command bars to put on dot required a lot more concentration 
SUBJECT 3: Rating of 4. Not exactly what I am used to flying and I don ’t have the luxury of having a PNF; 
SVS was significantly better than baseline 

SUBJECT 4: Rating of 3. Just not a display I would want to fly these approaches with. Lots of workload 
and very poor SA. 

SUBJECT 5: Rating of 4. I felt like I could make the approaches if I had to, but I sure wouldn 7 want to. 
SUBJECT 6: Rating of 4. Complicated approach relative to traditional approach and after flying the SVS 
concepts it redefines what is easy. Without seeing SVS, would have given different rating. 

SUBJECT 7: Rating of 5. Feel like playing catch up. Working the needles. 

SUBJECT 8: Rating of 5. You could do it, but wasn ’t very easy but not too bad. You don y t ' have some of the 
cue and can 7 see the terrain. 

SUBJECT 9: Rating of 4. Lack of precision. 

SUBJECT 10: 7. With the TAWS and VSD, it gives you some confidence with regard to the terrain and where 
the airplane is. 

SUBJECT 11: 8. As far as just the ease of flying, there is just not much to do. There are not a lot of step 
down maneuvers, etc. 

SUBJECT 12: 4. The hardest of all the ones that I saw today. 

SUBJECT 13: 3. The pitch did not seem to work very well except when you were stabilized. Had to do a lot 
of cross checking to get back ok. Occlusion dot should not be obscured by the command bars. Look at the 
777 PFD and the white box as example. 

SUBJECT 14: 3. Basically, the scan is slower and takes longer to interpret the terrain awareness. Gives 
you an idea of whats out there but you have to work to construct a 3D world with 2D information 
SUBJECT 15: 3. Symbology unfamiliar 
SUBJECT 16: 6. Have good vertical guidance over terrain. 

Missed approach? 

SUBJECT 1: Rating of 5. Same reason 
SUBJECT 2: Rating of 6. Same reason 
SUBJECT 3: Rating of 4. Same reason 
SUBJECT 4: Rating of 4. Same reason 
SUBJECT 5: Rating of 4. Same reason 

SUBJECT 6 Rating of 5. Not as much maneuvering so a little higher rating. But basically the same as 
approach 

SUBJECT 7: Rating of 4. A little harder. Single engine coming out of there had it been really engine out it 
would have been very difficult. 

SUBJECT 8: Rating of 6. The missed approach is almost procedural and if you follow those procedures. 

Doesn 7 make it very easy, but without the terrain depiction becomes labor intensive 
SUBJECT 9: Rating of 7. It was easier going up than coming down. 

SUBJECT 10: 6. Concern there was the turns I wasn 7 comfortable out near the end with the bright red 
TAWS 

SUBJECT 11: 5. There is a lot to do. You have to do a turn to immediately and there is no altitudes you had 
to watch leading to a 5 rating. 

SUBJECT 12 : 6. Easier. I wasn 7 as concerned about circle approach 

SUBJECT 13: 2. Speed on pitch and following the ED and if you follow it religiously would get big PIO 

SUBJECT 14: 3. Same Reason 

SUBJECT 15: 4. Symbology unfamiliar 

SUBJECT 16: 7. Liked having the speed guidance on the EADI. 
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2. Evaluate the ease of using the HUD photo-texture during the approach to rwy. 7 

SUBJECT 1: Rating of 7. Terrain features were great and provided good SA. Guidance got lost in the 
background 

SUBJECT 2: Rating of 6 although though generic was a little fuzzier than photo 

SUBJECT 3: Rating of 8. Excellent presentation of terrain and I could easily make out the runway and 
where I needed to go to make the approach 

SUBJECT 4: Rating of 7. With the exception of the fixed FOV, the HUD photo is an excellent display 
SUBJECT 5: Rating of 8. Very easy to fly. 

SUBJECT 6: Rating of 7. The ground makes all the difference in the world regardless of whether it is photo 
or generic 

SUBJECT 7: Rating of 8. Great SA. 

SUBJECT 8: Rating of 8 to a 9. Very easy. The only thing that made the HUD a little difficult was the 
sensitivity of the HUD (because of fixed FOV) 

SUBJECT 9: Rating of 7. It certainly more accurate but requires more flight tuning in terms of fight 
controls and eye-hand coordination. 

SUBJECT 10: 8. You didn ’t have to do any interpretation mentally. The picture was there. 

SUBJECT 11: 6. It seemed to me more sensitive to inputs and the way it was laid out doing the PIOs and 
also the FOV sometimes. 

SUBJECT 12: 7. Because SA higher, I could see everything in relation to everything. 

SUBJECT 13: 7. Roll control seem much more sensitive generally for the simulator 
SUBJECT 14: 8. Real 3D and your heads outside and minimizes heads down workload 
SUBJECT 15: 4 Following symbology is very sensitive. 

SUBJECT 16: 9. Problem was the sensitivity of the symbology. Trend indicatory really helped on the VV 

3. Evaluate the ease of using the HUD generic-texture during the approach to rwy. 7 

SUBJECT 1: Rating of 7. Same Reason as Q. 2 

SUBJECT 2: Rating of 6. Problems with pegged velocity vector which affected SA 
SUBJECT 3: Same 
SUBJECT 4: Same 
SUBJECT 5: Same 

SUBJECT 6: Rating 6. Photo had better depth perception 
SUBJECT 7: Rating of 8. Same reason. 

SUBJECT 8: Rating of 8 to 9. HUD generic actually in some cases was not easier to use but it had better 
contrast to it. With the better contrast, you could pick up features and weren ’t worried about shadows, etc. 
SUBJECT 9: Rating of 7. Same reason 
SUBJECT 10: Same and same reason. 

SUBJECT 11: Same and same reason 

SUBJECT 12: 8. 1 could see the nose relative to the terrain. 

SUBJECT 13: 6. Symbology was a little small for what should be on HUD; flashing / pegged velocity 
vector. Recommendation would be to keep it steady and use a dash or something to minimize attention 
capture. 

SUBJECT 14: 7. Your attention outside is a little longer to pick out the terrain features. 

SUBJECT 15: Same 

SUBJECT 16: Liked using the HUD and being able to look at versus look down a lot. 
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4. Evaluate the ease of performing a missed approach to rwy. 7 using the HUD generic-texture display concept 


SUBJECT 1: Rating of 7. Same as for Photo. No real perceived difference IN TERMS of “ ease of 
performing” 

SUBJECT 2: Rating of 6. Same as Q. 3 
SUBJECT 3: Same 
SUBJECT 4: Same 
SUBJECT 5: Same 

SUBJECT 6: Rating 6. Same as for Q3 
SUBJECT 7: Rating of 8. 

SUBJECT 8: Rating of 8 to 9. Very easy. 

SUBJECT 9: Rating of 8. Same reason 

SUBJECT 10: 8. Wasn y t all that too different. Generic presentation is good enough to give you a warm and 
fuzzy that terrain is not there. 

SUBJECT 11: Same 

SUBJECT 12: 5. Wasn ’t any information on there mostly because missing waterline and lots of distracting 
information. Really didn ’t do much for me. 

SUBJECT 13: 7. Same 
SUBJECT 14: Same 

SUBJECT 15: 5. Same problem. Not as difficult as approach. 

SUBJECT 16: 7. Same 

5. Evaluate the ease of performing a missed approach to rwy. 7 using the HUD photo-texture display concept 
SUBJECT 1: Rating of 2. “Meatball” provided very poor guidance 

SUBJECT 2: Rating of 7. Photo a little better than generic but needs work especially with FOV and losing 
terrain features when velocity vector gets on bottom of display 
SUBJECT 3: Same 
SUBJECT 4: Same 
SUBJECT 5: Same 

SUBJECT 6: Rating of 7 for same reason 
SUBJECT 7: Rating of 8. Excellent SA. 

SUBJECT 8: Same as photo 
SUBJECT 9: Rrating of 8. Same reason 
SUBJECT 10: Same. 

SUBJECT 11: Same 
SUBJECT 12: Same 
SUBJECT 13: Same 
SUBJECT 14: Same 
SUBJECT 15: Same 
SUBJECT 16: Same 
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6. Please comment on the perceived differences in using the HUD photo-texture and HUD generic-texture during 
the approaches to rwy. 7 

SUBJECT 1: None really. The photo terrain was sharper but both conveyed the same level of information 
needed to fly approach 

SUBJECT 2: Fuzzier concept; terrain wasn 7 that helpful on approached but helpful for missed. Provided 
good SA but not as good as HDD 

SUBJECT 3: Generic was really good for picking out those things I need to know where they are; however, 
it was not very good at helping me determine how far I was away from terrain 

SUBJECT 4: Photo realistic really increased SA compared to generic although generic is way better than 
what we have today. No question about it. 

SUBJECT 5: Really liked both to be honest although photo was easier to fly. Generic however was just a bit 
worse not really that much compared to photo. 

SUBJECT 6: Depth perception is superior in HUD Photo; 3D ability to differentiate above ground seem a 
little more realistic 

SUBJECT 7: Little to none. Slightly better definition on Photo. 

SUBJECT 8: Contrast was the big thing to me. I found sometimes that the HUD had much better contrast 
and I could discriminate things better. Photo sometimes I would lose the ghost. 

SUBJECT 9: The photo is much better in clarity, contrast, and I would even say topographical changes. It is 
easier to pick up the differences. I had a hard time with generic in judging distances 

SUBJECT 10: Although the HUD photo was more detailed, it didn 7 flow as smoothly. Maybe the graphics. 
The generic seemed more smoother. The photo texture was nice as it was very detailed. 

SUBJECT 11: The photo is more scarier to fly as you start to see more of the terrain; it is “ more realistic 

SUBJECT 12: The HUD photo has more important information presented the only bad thing about HUD 
photo was sometimes you could use the velocity vector or flight director (didn 7 have control over 
brightness and contrast). Important information includes ridge line, etc. 

SUBJECT 13: Generic is fine when you are higher up. Photo is very essential when you get into areas that 
are close to the terrain. Had a much more comfortable feel when you are really high up. Need to have 
dual databases or separate sources for EGPWS and SVS to minimize complacency. 

SUBJECT 14: The generic takes longer to pick out key terrain features although simple acquisition of 
features like rivers, etc. 

SUBJECT 15: I thought the photo just having that texture in the background gave me a lot more confidence 
in relation to where I was with respect to terrain. 

SUBJECT 16: Always liked the photo better because it was crisper. 


7. Please rate which NASA HUD display concept (e.g., generic) provided the best level of situation awareness in 
performing the missed approach to rwy. 7. (Situation awareness defined as: 64 . . .the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or non-normal 
conditions.”). 


SUBJECT 1: 
SUBJECT 2: 
SUBJECT 3: 
SUBJECT 4: 
SUBJECT 5: 
SUBJECT 6: 
SUBJECT 7: 
SUBJECT 8: 
SUBJECT 9: 
SUBJECT 10: 
SUBJECT 11: 
SUBJECT 12: 
SUBJECT 13: 
SUBJECT 14: 
SUBJECT 15: 
SUBJECT 16: 


Rating of Photo 

Rating of Photo 

Photo 

Photo 

Photo 

Photo 

Photo. 

Generic 

Photo 

Photo. 

Equal 
HUD Photo 
HUD Photo 
HUD Photo 
HUD Photo 
HUD Photo 
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Please elaborate on your answer and discuss the reason for your choice of HUD display concept 


SUBJECT 1: The HUD Photo was a little sharper and easier to fly because it provided more details (i.e., 
texture gradients) 

SUBJECT 2: Texture was much better; with generic relied on instruments whereas with Photo could use 
terrain depiction more 

SUBJECT 3: Just a little better than generic although I liked both a lot. 

SUBJECT 4: Generic was a bit washed out and hard to really tell where you were relative to terrain. 
However, if I stay with published approach procedures, generic is excellent in giving me tactical feedback 
that I am ok. 

SUBJECT 5: Photo was better but not much better. Gave details that I would like to have if cost was not a 
consideration. 

SUBJECT 6: See #6 above 

SUBJECT 7: More easily scanned and interpreted what is out there. Just the generic doesn ’t give as much 
but it is not as detailed but little real difference 

SUBJECT 8: Contrast but you liked the photo because it looked real. Looked like it should. But caveat that, 
how does it look like when we break out how does the real world look like. 

SUBJECT 9: Sharper and I could judge distance better. 

SUBJECT 10: Liked the photo because you could pick out roads and other cultural features especially with 
approach with requires visual contact with featuers. 

SUBJECT 11: Both provide an adequate amount of SA 

SUBJECT 12: Just gave so much more information than the generic concept 

SUBJECT 13: It was nice. It gives you much more SA there is certain things that are easily identifiable. 
Generic features should be put in to make it easier to pick out key cultural, etc. features. 

SUBJECT 14: Provides a better picture of surrounding terrain; shade of grey comparison. HUD generic 
was close to Photo but photo did give you better SA and more confidence 

SUBJECT 15: Just the presentation. The Photo was better in terms of terrain depiction and I could better 
trust the presentation. 

SUBJECT 16: Crisper. Better resolution. More confidence. 
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8. Please discuss whether how the use of a tunnel (without flight director) would affect the operational safety and 
use of the NASA HUD display concept for approaches to rwy. 7 

SUBJECT 1: It was nice to have just in case you got out of the tunnel 

SUBJECT 2: Sense of that I knew I was ok if I was still in tunnel 

SUBJECT 3: Tunnel was great. 

SUBJECT 4: Tunnel helped you know that you were still on the path and even if you were not right on the 
guidance, the tunnel provided that sense of safety comfort that you don ’t normally get with traditional 
instruments and displays 

SUBJECT 5: Tunnel could be of big help for RNP and curved approaches. 

SUBJECT 6: The only thing I noticed about the tunnel is that the tunnel started to tunnel before I started to 
turn so I thought I was drifting but got used to it 

SUBJECT 7: I think lateral deviations would greater in number 

SUBJECT 8: I liked the tunnel. The tunnel was good because it provided a very nice manageable trend 
information. I could see ahead and help with staying ahead of the aircraft. 

SUBJECT 9: It didn ’t detract but didn ’t find it helpful 

SUBJECT 10: Tunnel was good for the turns and to know they are coming. Would like to have distance to 
go to next waypoint. 

SUBJECT 11: 1 don ’t think it really did much at all because you don ’t’ have the different colors so it didn ’t 
stand out that much. Didn ’t use the tunnel. 

SUBJECT 12: Liked it. But it was much better PFD than HUD. It was distracting on the HUD and didn ’t 
use it as much in the HUD. 

SUBJECT 13: Its there and it is kinda of a peripheral and I looked at it a few times but I think you are mostly 
focused on ghost and the tunnel is a nice adjunct. If you are outside the tunnel to find your way back to the 
path. 

SUBJECT 14: Liked it. What I liked about you could see ahead of the tunnel that the path was going to turn; 
good anticipation. 

SUBJECT 15: Not as helpful on the HUD as HDD. Contrast in not as good in the HUD. Magenta better 
stands out in the HDD. There is so much busy work on the HUD that the tunnel doesn ’t show up as well. 

SUBJECT 16: In the beginning, I didn ’t use it but towards the end I was using it a great deal. I finally 
realized that the tunnel gave me a respresentation of the radius of turn which I couldn ’t get from the ghost. 
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9. Please provide any additional comments that would be helpful in evaluating the HUD display concept 

SUBJECT 1: None 
SUBJECT 2: None 
SUBJECT 3: Excellent display. 

SUBJECT 4: Concern about fixed FOV and sensitivity ofFMA and other guidance 
SUBJECT 5: Velocity vector really sensitive; a lot of workload 

SUBJECT 6: Would give up the tunnel before ghost; really liked HUD concept but concerned about fixed 
FOV especially in turns and high AoA with the velocity vector getting pegged and losing terrain depiction 
SUBJECT 7: Really like it. Clear, concise, easily interpreted. 

SUBJECT 8: None 

SUBJECT 9: The HUD is my preference for one particular reason for the OTW transition. 

SUBJECT 10: ghost I didn ’t ’ really like. It was much more aggressive than I would have flown and wasn ’t 
as smooth as the command bars 

SUBJECT 11: May get a lot of PIO when going. VSI missing. Everything I need wasn ’t all presented on the 
HUD and had to cross-check it inside. 

SUBJECT 12: The only thing I didn ’t like is with the missed approach there is no way to get any peripheral 
information in the HUD once it gets pegged with just the doughnut. 

SUBJECT 13: I actually don ’t know the value of the waterline I know what it is there but I would assume 
have had an AoA indicator than the waterline and it got in the way. 

SUBJECT 14: The HUD in general, the FOV is too limited and it gets pegged and makes it difficult to 
follow FD cues on GA. 

SUBJECT 15: I thought the HUD was the most difficult to fly with the exception of the baseline. It would be 
better than A if the sensitivity of the symbology was improved. Fixed FOV was difficult to fly. 

SUBJECT 16: Excellent ! 

10. Evaluate the ease of predicting flight path using the “follow me aircraft” 

SUBJECT 1: 8. SUBJECT stated that it provided excellent “look ahead” 

SUBJECT 2: 8. Good preview. Little aggressive in turns 

SUBJECT 3: 8. With the exception of when it turned, the FMA was easily to understand and follow 
SUBJECT 4: Almost too easy. 

SUBJECT 5: Great guidance aid; I felt that, once I was comfortable with the display, I could easily manage 
the approach using the FMA 

SUBJECT 6: 8. Seemed intuitive; it was nice to follow around in the sky; was in the bank turn; visual angle 
is large instead of looking 70 degrees and should be smaller to reflect; almost fills up too much of the 
screen 

SUBJECT 7: 8. Only gripe with the ghost aircraft has no roll motion just yaw motion. Somewhat unsettling. 
Can adapt it. Visual perception aspect angle. 

SUBJECT 8: 8. Easy. Good prediction with the ghost. I was using the ghost and the noodle together to see 
the track line. Sometimes the ghost sometimes you would roll inside and if noodle says I’m ok then I didn ’t 
have to work about keeping ball on ball. 

SUBJECT 9: 8. Great way to maintain flight path guidance 

SUBJECT 10: 8. I did find it somewhat distracting at times but I think overall it was very helpful in helping 
me feel comfortable that I was ok and on the path. See above on comments about ghost and being 
aggressive. 

SUBJECT 11: I thought it was helpful and I used it. It gave the direction to follow and it showed turns, etc. 
SUBJECT 12: Really liked it a lot. Liked the 3D of the triangle and gives you the rate and trend of the path 
and could really fly the approach with VV and ghost. 

SUBJECT 13: I liked it but I would reimplement it. I would make bank angle parallel so that the wings of 
the VV are parallel to the wings of the ghost to denote the bank angle desired 
SUBJECT 14: I liked it. It provides good trend information. 

SUBJECT 15: The ghost was very good in terms of predictive information and I found it very easy to fly. 
SUBJECT 16: Liked that a lot. Great guidance and made the workload much easier. 


74 



1 1 . Please provide your comments on the use of the tunnel in the NASA HUD display concept. 


SUBJECT 1: Stated in Q. 8 
SUBJECT 2: None 
SUBJECT 3: None. 

SUBJECT 4: I liked it although you would have to supply a declutter option 
SUBJECT 5: None. 

SUBJECT 6: I like it. You can get to carried away so likes the minimized clutter; you can forget you are in 
airplane 

SUBJECT 7: Greatly enhance ability to fly approach and stay in the parameters 
SUBJECT 8: Really like that. 

SUBJECT 9: see above. 


SUBJECT 10: Great concept. Liked that it wasn ’t too much in your face. 

SUBJECT 11: Didn ’t really use it but could be useful for RNP and sense of staying on path. 

SUBJECT 12: It was good that it was magenta for PFD but didn y t think it was that great for HUD. The 
distance and spacing was great. 

SUBJECT 13: See above 

SUBJECT 14: It was fine. It wasn y t distracting and it is much better than boxes like you see in MS flight sim. 
SUBJECT 15: see above. 


SUBJECT 16: See above. 


12. During the approaches to rwy 7 using the NASA SVS HUD, please comment on the use of de-clutter options 
you would have liked to have available. 

SUBJECT 1: An absolute must have. The de-clutter options need to be included because of the terrain you 
can y t see through. I have to be sure that the terrain depiction is right. Also, when I don ’t need it, I don ’t 
want to have it up there. 

SUBJECT 2: Would have decluttered the HUD on first 20 degree turn. On break out as well when reached 
DH 

SUBJECT 3: Must have. I would declutter on break out and to check HUD SVS Terrain with OTW. 

SUBJECT 4: None. 

SUBJECT 5: Very rarely, actually never, are you going to be in 0,0 approach so it is hard to answer 
question. I liked to have control available to me so yes I would prefer to declutter and would use that 
feature to check integrity of HUD terrain depiction with outside world. 

SUBJECT 6: If I had to turn off anything I would not have had to turn off anything. 0/0 so seldom but it 
might be enough to be able to adjust the brightness and contrast without having to declutter 

SUBJECT 7: HUD didn y t see a need to declutter. 

SUBJECT 8: On break out would like to declutter and some symbology in the turns. 

SUBJECT 9: Would have gotten rid of the tunnel. Other than that I would leave on although the use of 
terrain on approach is limited. 

SUBJECT 10: I think I would like to declutter it to check the real world terrain. Iam a little uncomfortable 
with it being opaque especially if it isn y t conformal to the outside world which it doesn y t seem to be. 

SUBJECT 11: Wouldn y t declutter. Would like to add stuff. 

SUBJECT 12: Like to declutter only in VMC so I would like to leave everything up. Would like VSI. 

SUBJECT 13: No. HUD clutter doesn y t bother me although perhaps taking off terrain to verify in less than 
CAT III. I would have occlusion logic to make it a priority that the ghost and VV are primary and other 
symbology is secondary. Bank angle should be closer to help with bank angle information. F-16 example 
puts the bank angle cues on the VV. 

SUBJECT 14: In C130J at night approach and get the runway lights, the HUD tends to get washed out. On 
final approach phase. On final landing segment, keep key symbology. Would like VSI heads-up. 

SUBJECT 15: Symbology size was too small in all concepts. Declutter options at EGE would be good but 
really contrast and brightness controls easily accessible would be great. 

SUBJECT 16: on breakout would use declutter and decent weather. 
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13. Evaluate the ease of using the tunnel for flight path guidance 
GENERIC TEXTURE 

SUBJECT 1: Rating of Size A: 8 and Size X: 9 
SUBJECT 2: Rating of Size A: 7 and Size X: 7 
SUBJECT 3: A: 7; X: 9 
SUBJECT 4: A: 7; X: 8 
SUBJECT 5: A: 7; X: 8 
SUBJECT 6: Rating of Size A: 6 and Size X: 8 
SUBJECT 7: Rating of Size A: 6 and Size X: 9 
SUBJECT 8: Rating of Size A: 6 and Size X: 8 
SUBJECT 9: Rating of Size A: 4 and Size X: 4 
SUBJECT 10: Rating of Size A: 6 and Size X: 7 
SUBJECT 11: Rating of Size A: 6 and Size X: 8 
SUBJECT 12: Rating of Size A: 6 and Size X: 8 
SUBJECT 13: 6 for size A and 6 for Size X 
SUBJECT 14: Rating of 6 for Size A and 8 for Size X 
SUBJECT 15: 7 for A and 8 for X 
SUBJECT 1 6: 7 for A and 8 for X 

PHOTO TEXTURE 

SUBJECT 1: Rating of Size A: 9 and Size X: 9 
SUBJECT 2: Rating of Size A: 7 and Size X: 7 
SUBJECT 3: Same 
SUBJECT 4: Same 
SUBJECT 5: Same 

SUBJECT 6: Rating of Size A: 7 and Size X: 9 
SUBJECT 7: Rating of Size A: 6 and Size X: 9 
SUBJECT 8: Rating of Size A: 7 and Size X: 9 
SUBJECT 9: Rating of Size A: 4 and Size X: 4 
SUBJECT 10: Rating of Size A: 5 and SizeX: 7 
SUBJECT 11: Rating of Size A: 6 and SizeX: 8 
SUBJECT 12: Size A: 7 and Size X: 9 
SUBJECT 13: Size A: 6 and SizeX: 6 
SUBJECT 14: Size A: 7 and SizeX: 7 
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14. Please provide your comments on the use of the tunnel in the NASA HDD display concept. 

SUBJECT 1: May have been too many frames when the angle irons appeared. Reduce the rate of the flow of 
angle irons 

SUBJECT 2: Was helpful if I was not on ghost aircraft. A little too minimal but may be good for long flights. 

SUBJECT 3: Same as HUD 

SUBJECT 4: Same as HUD 

SUBJECT 5: Same as HUD 

SUBJECT 6: Same thing as NASA HUD tunnel as long as it doesn ’t overpower all the other things you are 
trying to look at it. Good reference cue in addition to other guidance is great but not by itself. 

SUBJECT 7: Mostly size. The size A even changing 30, 60, 90 and Unity didn ’t give the same impression 
and had to keep switching in the X. 

SUBJECT 8: The Size X Photo was just great. It gave me all the information I needed. However, I would say 
that the Generic texture gave much better contrast and I could see how it would be an issue on a day when 
lam flying in the snow and the photo picture looks nothing like the outside world. At least with Generic, I 
know it is artificial. 

SUBJECT 9: Just didn ’t need it with the terrain and ghost aircraft. 

SUBJECT 10: The tunnel was ok but the small size on A made it difficult to see other symbology and I really 
didn ’t need it much since I had the ghost. 

SUBJECT 11: The FOV of size A made the tunnel difficult to use. Also, I really didn ’t need to use the tunnel 
much except with Size A because of difficulty of small FOV needed to see symbology yet increased the 
chance of getting outside the tunnel. 

SUBJECT 12: Same as for HUD. 

SUBJECT 13: The reduced clutter of the HDD was great and it provided just enough detail and look ahead 
to know when I was going into a turn. Would have liked a TOGA tunnel 

SUBJECT 14: Same as HUD 

SUBJECT 15: TOGA Tunnel would have helped. 

SUBJECT 16: No comments. Liked the tunnel and thought it was helpful but the SVS terrain and FMA was 
all I really needed. 

15. Please provide any comments on the symbology used in the NASA HDD display concept 


SUBJECT 1: None 

SUBJECT 2: heading indicator numbers were squished and fuzzy 

SUBJECT 3: No. 

SUBJECT 4: No. 

SUBJECT 5: Wasn ’t really like the 757 instruments since there is a lot of symbology missing from your 
display and arranged differently than on the EADI; however, I got used to it and was able to fly what you 
call the baseline. 

SUBJECT 6: No 

SUBJECT 7: Size A was too cluttered . Even though it wasn ’t integrated it was way too much information on 
the SVS display. Would like to declutter HDD or dim. 

SUBJECT 8: Size A was way too cluttered. But, I couldn ’t use a low FOV because I could not see enough 
terrain. 

SUBJECT 9: Tunnel on Size A was distracting. 

SUBJECT 10: None 


SUBJECT 11: Size A with larger FOV made the symbology too cluttered (display) 

SUBJECT 12: really liked the ghost aircraft and could picture it 3D. Liked runway highlight and extended 
centerline 


SUBJECT 13: 
SUBJECT 14: 
SUBJECT 15: 
SUBJECT 16: 


A bank angle cue on the ghost would be helpful 
None 

Not really like what I would expect to see for baseline symbology. 

Symbology was different that I am used to seeing although it wasn ’t hard to fly. 
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16. Please rate the workload associated using each NASA display concept for the Rwy. 07 approach . We can 
define workload as: “the degree of cognitive processing capacity required to perform the flight task approach 
adequately” 

These are reversed scored with high numbers = lower workload 


GENERIC TEXTURE 

SUBJECT 1: Rating of 7 for Size A and 8 for Size X 
SUBJECT 2: Rating of 7 for Size A and 8 for Size X 
SUBJECT 3: Rating of 7 for Size A and 9 for Size X 
SUBJECT 4: Rating of 7 for Size A and 8 for Size X 
SUBJECT 5: Rating of 6 for Size A and 8 for Size X 
SUBJECT 6: Rating of 5 for Size A and 7 for Size X 
SUBJECT 7: Rating of 7 for Size A and 8 for Size X 
SUBJECT 8: Rating of 7 for Size A and 8 for Size X 
SUBJECT 9: Rating of 7 for Size A and 8 for Size X 
SUBJECT 10: Rating of Size A: 7 and Size X was 8 
SUBJECT 11: Rating of Size A: 7 and Size X was 9 
SUBJECT 12: Rating of Size A: 5 and Size X was 6 
SUBJECT 13: Rating of Size A: 6 and Size X was 7 
SUBJECT 14: Rating of 7 for A and 8 for X 
SUBJECT 15: Rating of 6 for A and 9 for X 
SUBJECT 1 6: Rating of 7 for A and 9 for X 


PHOTO TEXTURE 

SUBJECT 1: Rating of 7 for Size A and 8 for Size X 

SUBJECT 2: Rating of 8 for Size A and 8 for Size X 

SUBJECT 3: 8 for A and 9 for X 

SUBJECT 4: 6 for A and 9 for X 

SUBJECT 5: 7 for A and 9 for X 

SUBJECT 6: Rating of 6 for Size A and 8 for Size X 

SUBJECT 7: Same ratings. 

SUBJECT 8: Same 
SUBJECT 9: Same 

SUBJECT 10: Rating of Size A: 7 and Size X: 8 
SUBJECT 11: Same 

SUBJECT 12: Size A was 6 and Size X was 8 
SUBJECT 13: Size A was 7 and Size X was 9 
SUBJECT 14: Same 
SUBJECT 15: 8 for A and 8 for X 
SUBJECT 16: 6 for A and 9 for X 


BASEL INE 

SUBJECT 1: Rating of 4 
SUBJECT 2: Rating of 5 
SUBJECT 3: 5 
SUBJECT 4: 6 
SUBJECT 5: 6 
SUBJECT 6: Rating of 3 
SUBJECT 7: Rating of 4 
SUBJECT 8: Rating of 4 
SUBJECT 9: Rating of 3 
SUBJECT 10: Rating of 3 
SUBJECT 11: Rating of 4 
SUBJECT 12: Rating of 3 
SUBJECT 13: Rating of 3 
SUBJECT 14: 3 
SUBJECT 15: 4 
SUBJECT 16: 6 


Approach and Missed 


HUD 

Photo 

HUD 

Generic 

7 

7 

6 

6 

8 

8 

7 

7 

8 

8 

7 

6 

8 

8 

9 

9 

7 

7 

8 

8 

6 

6 

7 

8 

7 

6 

8 

7 

4 

4 

9 

9 
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Please rate the workload associated using each NASA display concept for the Rwy. 07 missed approach . We can define 
workload as: “the degree of cognitive processing capacity required to perform the flight task approach adequately” 

Reversed Scored 


GENERIC TEXTURE 

SUBJECT 1: Rating of 7 for Size A and 8 for Size X 
SUBJECT 2: Rating of 7 for Size A and 8 for Size X 
SUBJECT 3: Rating of 7 for Size A and 9 for Size X 
SUBJECT 4: Rating of 6 for Size A and 7 for SizeX 
SUBJECT 5: Same as for approach 
SUBJECT 6: Same as for approach. 

SUBJECT 7: Same 
SUBJECT 8: Same 
SUBJECT 9: Rating of 5. 

SUBJECT 9: Same 
SUBJECT 10: Same 
SUBJECT 11: Same 
SUBJECT 12: Same 
SUBJECT 13: Same 
SUBJECT 14: Same 
SUBJECT 15: Same 
SUBJECT 16: Same 


PHOTO TEXTURE 

SUBJECT 1: Rating of 7 for size A and 8 
for Size X 

SUBJECT 2: Rating of 8 for Size A and 8 
for Size X 

SUBJECT 3: Rating of 7 for Size A and 8 
for Size X 

SUBJECT 4: Same as approach 
SUBJECT 5: Same 
SUBJECT 6: Same 
SUBJECT 7: Same 
SUBJECT 8: Same 
SUBJECT 9: Same 
SUBJECT 10: Same 
SUBJECT 11: Same 
SUBJECT 12: Same 
SUBJECT 13: Same 
SUBJECT 14: Same 
SUBJECT 15: Same 
SUBJECT 16: Same 


BASEL INE 

SUBJECT 1: Rating of 4 
SUBJECT 2: Rating of 4 
SUBJECT 3: Rating of 4 
SUBJECT 4: Rating of 4 
SUBJECT 5: Rating of 4 
SUBJECT 6: Rating of 4 
SUBJECT 7: Rating of 4 
SUBJECT 8: Rating of 5 
SUBJECT 9: Rating of 2 
SUBJECT 10: Rating of 2 
SUBJECT 11: Rating of 3 
SUBJECT 12: Rating of 3 
SUBJECT 13: Rating of 3 
SUBJECT 14: Rating of 4 
SUBJECT 15: Rating of 4 
SUBJECT 16: Rating of 4 
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16. Please evaluate the level of situation awareness experienced during the approach to rwy. 7 for each display 
concept. We define situation awareness as: 64 . . .the pilot has an integrated understanding of the factors that will 
contribute to the safe flying of the aircraft under normal or non-normal conditions.” 


Reversed Scored 

GENERIC TEXTURE 

SUBJECT 1: Rating of 8 for Size A and 9 for Size X 
SUBJECT 2: Rating of 7 for Size A and 9 for Size X 
SUBJECT 3: Rating of 7 for Size A and 9 for Size X 
SUBJECT 4: Rating of 7 for Size A and 8 for Size X 
SUBJECT 5: Rating of 6 for Size A and 9 for Size X 
SUBJECT 6: Rating of 7 for Size A and 8 for Size X 
SUBJECT 7: Rating of 8 for Size A and 9 for Size X 
SUBJECT 8: Rating of 7 for Size A and 9 for Size X 
SUBJECT 9: Rating of 8 for Size A and 9 for Size X 
SUBJECT 10: Rating of Size A: 7 and 9 for SizeX 
SUBJECT 11: Rating of Size A: 7 and 9 for Size X 
SUBJECT 12: Rating of Size A: 7 and 9 for Size X 
SUBJECT 13: Rating of Size A: 7 and 8 for Size X 
SUBJECT 14: Rating of 6 for Size A and 7 for Size 
X 

SUBJECT 15: Rating of 6 for Size A and 8 for Size 
X 

SUBJECT 16: Rating of 7 for Size A and 9 for Size 
X 


PHOTO TEXTURE 

SUBJECT 1: Rating of 8 for Size A and 9 for SizeX 
SUBJECT 2: Rating of 7 for Size A and 9 for Size X 
SUBJECT 3: Rating of 7 for Size A and 9 for Size X 
SUBJECT 4: Rating of 7 for Size A and 9 for Size X 
SUBJECT 5: Rating of 7 for Size A and 9 for Size X 
SUBJECT 6: Rating of 8 for Size A and 9 for Size X 
SUBJECT 7: Rating of 8 for Size A and 9 for Size X; 

really didn ’t jump out as a difference 
SUBJECT 8: Rating of 7 for Size A and 9 for Size X 
SUBJECT 9: Rating of 7 for Size A and 9 for Size X 
SUBJECT 10: Rating of 7 for Size A and 8 for Size 
X 

SUBJECT 11: Rating of 7 for Size A and 9 for Size 
X 

SUBJECT 12: Rating of 7 for Size A and 8 for Size 
X 

SUBJECT 13: 7 for Size A and 9 for Size X 
SUBJECT 14: Rating of 7 for Size A and 9 for Size 
X 

SUBJECT 15: Rating of 6 for Size A and 8 for Size 
X 

SUBJECT 16: Rating of 7 for Size A and 9 for Size 
X 


BASELINE 

SUBJECT 1: Rating of 4 
SUBJECT 2: Rating of 3 
SUBJECT 3: Rating of 4 
SUBJECT 4: Rating of 4 
SUBJECT 5: Rating of 4 
SUBJECT 6: Rating of 5 
SUBJECT 7: Rating of 4 
SUBJECT 8: Rating of 3 
SUBJECT 9: Rating of 4 
SUBJECT 10: Rating of 4 
SUBJECT 11: Rating of 4 
SUBJECT 12: Rating of 4 
SUBJECT 13: Rating of 4 
SUBJECT 14: Rating of 4 
SUBJECT 15: Rating of 5 
SUBJECT 16: Rating of 3 
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17. Please evaluate the level of situation awareness experienced during the missed approach to rwy. 7 for each 
display concept. We define situation awareness as: 64 . . .the pilot has an integrated understanding of the factors 
that will contribute to the safe flying of the aircraft under normal or non-normal conditions.” 


GENERIC TEXTURE 
SUBJECT 1: Rating of 8 for Size A and 

PHOTO TEXTURE 
SUBJECT 1: Rating of 8 

BASELINE 

9 for Size X 

for Size A and 9 for Size 

SUBJECT 1: Rating of 4 

SUBJECT 2: Rating of 7 for Size A and 

X 

SUBJECT 2: Rating of 2 

9 for Size 

SUBJECT 2: Rating of 7 

SUBJECT 3: Rating of 2 

SUBJECT 3: Rating of 7 for Size A and 

for Size A and 9 for Size X 

SUBJECT 4: Rating of 4 

8 for Size 

SUBJECT 3: Same as 

SUBJECT 5: Rating of 3 

SUBJECT 4: Rating of 6 for Size A and 

approach 

SUBJECT 6: Rating of 2 

8 for Size 

SUBJECT 4: Same 

SUBJECT 7: Rating of 2 

SUBJECT 5: Rating of 7 for Size A and 

SUBJECT 5: Same 

SUBJECT 8: Rating of 2 

9 for Size 

SUBJECT 6: Same 

SUBJECT 9: Rating of 2 

SUBJECT 6: Same as approach 

SUBJECT 7: Same 

SUBJECT 10: Rating of 2 

SUBJECT 7: Same 

SUBJECT 8: Same 

SUBJECT 11: Rating of 2 

SUBJECT 8: Same 

SUBJECT 9: Same 

SUBJECT 12: Rating of 2 

SUBJECT 9: Same 

SUBJECT 10: Same 

SUBJECT 13: Rating of 3 

SUBJECT 10: Same 

SUBJECT 11: Same 

SUBJECT 14: Rating of 2 

SUBJECT 11: Same 

SUBJECT 12: Same 

SUBJECT 15: Rating of 3 

SUBJECT 12: Same 

SUBJECT 13: Same 

SUBJECT 16: Rating of 3 

SUBJECT 13: Same 
SUBJECT 14: Same as approach 
SUBJECT 15: Same as approach 
SUBJECT 16: Same 

SUBJECT 14: Same 
SUBJECT 15: Same 
SUBJECT 16: Same 



HUD 

Photo 

HUD 

Generic 

8 

6 

8 

6 

8 

8 

7 

7 

9 

6 

7 

6 

8 

8 

9 

6 

7 

7 

8 

8 

6 

7 

8 

8 

7 

6 

8 

7 

6 

4 

9 

7 
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18. Please provide any comments that would be helpful to us in evaluating the difference terrain-textures options 
for the display concept 

SUBJECT 1: Preferred photo but generic was acceptable. Definitely better than baseline 
SUBJECT 2: Photo was better but the lack of peripheral cues was a negative 
SUBJECT 3: Photo was superior although the generic did let you see things faster like rivers 
SUBJECT 4: Picking out cultural features was easier with generic although photo was much better to look 
at 

SUBJECT 5: You should look at combining the best features of both! 

SUBJECT 6: None 

SUBJECT 7: The photo allowed for normal expectations of what the normal terrain would be. Had to 
interpret the color codings but was minimal. 

SUBJECT 8: Contrast was much better with Generic. 

SUBJECT 9: Same as for HUD 

SUBJECT 10: The generic made it easier to see features like a river or the valley floor. 

SUBJECT 11: Photo was really nice, but I could imagine that generic would be better for quickly locating 
key terrain or cultural features especially with EGE since it is a visual contact approach. 

SUBJECT 12: The Photo has so much better pixel quality that it really gives you a lot of confidence that you 
were safe and knew exactly where the terrain is. But, the generic was pretty good and if cost was a factor 
then it would be acceptable and much better than EADI. 

SUBJECT 13: Depth perception was not as good with Generic texture. 

SUBJECT 14: Depth perception and lack of cues with generic 
SUBJECT 15: Both were very good although the photo was my choice 

SUBJECT 16: Just liked the photo better. It had more cues and I felt like I could really use the display to 
ensure terrain separation. The generic is good if you stayed with published approach but not as good if you 
got into trouble. Both are very much better than baseline which was not good at all relatively speaking. 

19. Please rate which display concept you would prefer to use to make approach to rwy. 7 

SUBJECT 1: Size A Photo, Size X Photo, Size X photo, Size X generic, and Size X generic. Liked the larger 
size better than Photo. 

SUBJECT 2: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size better 
than Photo. 

SUBJECT 3: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size better 
than Photo. 

SUBJECT 4: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size better 
than Photo. 

SUBJECT 5: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size better 
than Photo. 

SUBJECT 6: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size better 
than Photo 

SUBJECT 7: Size A Photo, Size X Photo, Size X Photo, Size X generic, Size X generic. Photo versus generic 
minimal 

SUBJECT 8: Size A Photo, Size X Photo, Size X Photo, Size X generic, Size X generic. Size and generic was 
no contest to Size A photo. 

SUBJECT 9: Size A photo, size x photo, size x photo, size x generic, size x generic. Size major factor. 
SUBJECT 10: Size A photo, size x photo, size x photo, size x generic, size x generic. Size major factor 

SUBJECT 11: Size A photo, Size x photo, size x generic, size x generic, size major fact 

SUBJECT 12: Size A photo, size x photo, size x generic, size x generic, size major factor. 

SUBJECT 13: Size A photo, size x photo, size x generic, size x generic, size major factor 

SUBJECT 14: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size 
better than Photo. 

SUBJECT 15: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size 
better than Photo. 

SUBJECT 16: Size A Photo, Size X Photo, Size X Photo, Size X generic, and Size X generic. Large size 
better than Photo. 


82 



VERTICAL SITUATION DISPLAY 


20. On a scale of 1 to 10, please rate how much your situation awareness was increased with the vertical situation 
display provided during your approach to rwy. 7 using the baseline EADI. 

Approach rwy 07 using the baseline EADI Missed Approach rwy. 07 using the baseline EADI 


SUBJECT 1: Rating of 5 

SUBJECT 1: Rating of 5 

SUBJECT 2: Rating of 6 

SUBJECT 2: Rating of 5 

SUBJECT 3: Rating of 5 

SUBJECT 3: Rating of 5 

SUBJECT 4: Rating of 5 

SUBJECT 4: Rating of 6 

SUBJECT 5: Rating of 5 

SUBJECT 5: Rating of 6 

SUBJECT 6: Rating of 6 

SUBJECT 6: Rating of 8 

SUBJECT 7: Rating of 6 

SUBJECT 7: Rating of 6 

SUBJECT 8: Rating of 8 

SUBJECT 8: Rating of 8 

SUBJECT 9: Rating of 0 

SUBJECT 9: Rating of 0 

SUBJECT 10: Rating of 6 

SUBJECT 10: Rating of 6 

SUBJECT 11: Rating of 10 

SUBJECT 11: Rating of 10 

SUBJECT 11: Rating of 6 

SUBJECT 11: Rating of 6 

SUBJECT 12: Rating of 9 

SUBJECT 12: Rating of 5 

SUBJECT 13: Rating of 8 

SUBJECT 13: Rating of 5 

SUBJECT 14: Rating of 7 

SUBJECT 14: Rating of 6 

SUBJECT 15: Rating of 7 

SUBJECT 15: Rating of 8 

SUBJECT 16: Rating of 5 

SUBJECT 16: Rating of 6 


Please provide your reason for the rating provided below: 

SUBJECT 1: Iwasn ’ t that comfortable with using it, but it definitely enhanced the TAFTS since it was a red 
box on the go-around when got close to the terrain. The VSD allowed me to be sure that I was ok. 

However, I still was uncomfortable and there is no way I would rely on that in my aircraft alone. I would 
have been really uncomfortable with what I was seeing in the baseline condition. 

SUBJECT 2: The VSD gave some indication that the TA WS was exactly showing me the right information. I 
did notice that I had to be careful and look at my ND range and think about where the red box from the 
HSI EGPWS was on the vertical ND. 

SUBJECT 3: The VSD gave me a profile view of the TAWS which was not very helpful on the miss because it 
was always showing me that I was in danger of hitting terrain. The VSD on the other hand let me know that 
I was ok and that the notch was still there. 

SUBJECT 4: I didn ’t use it as much as I probably should have. With practice, I can see the VSD being a 
great cross-check instrument. 

SUBJECT 5: VSD was not help at all on approach, but on the miss it did give me an indication that what the 
TA WS was telling (and not telling me) was correct. However, not even close a comparison with SVS. 

SUBJECT 6: For approach it helps a little, but the missed really helps to minimize the impact of the red 
squares. 

SUBJECT 7: With the baseline, it gives some immediate SA to determine if I am going to tie the “ low level 
record” 

SUBJECT 8: The VSD helps significantly with the baseline. It just gives different information that I can use 
with regard to terrain out there. The TA WS is nice but it was flashing red at the end of the miss and I could 
use the VSD to say to myself that I had sufficient terrain clearance. 

SUBJECT 9: The VSD provides nothing the TA WS doesn ’t. 

SUBJECT 10: The VSD gave a correlation to the TAWS to help cross-reference. It gave a view of what 
terrain was right in front of the airplane. 

SUBJECT 11: You were flying right into red and therefore it helped to confirm what you knew from the 
training that there was notch out there that the TA WS system wasn ’t picking up 

SUBJECT 12: Single source of terrain information from TA WS and VSD allows cross-checking and backup 
of terrain information 

SUBJECT 13: There is no way to know with TA WS that you are ok the VSD gave a “ huge help ”. 

SUBJECT 14: Another piece of information to help build a 3D model of what is out there, the VSD adds 
considerably to the TA WS. The VSD gives more of a gradient of the terrain more so than the TA WS 
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SUBJECT 15: Rating would have been higher perhaps had I been able to use it but I was so focused on the 
command bars with baseline that I wasn ’t able to take full advantage of the display and what it could offer 
me. 

SUBJECT 16: Did help. The GPWS displays are integrated like these are and that is why it helps more than 
say no help at all. 

21. On a scale of 1 to 10, please rate how much your situation awareness was increased with the vertical situation 
display provided during your approach to rwy. 7 using the SYS Size X. 


Approach rwv 07 using Size X 

Missed Approach rwv. 07 using Size X 

SUBJECT 1: Rating of 0 

SUBJECT 1: Rating of 0 

SUBJECT 2: Rating of 2 

SUBJECT 2: Rating of 2 

SUBJECT 3: Rating of 0 

SUBJECT 3: Rating of 0 

SUBJECT 4: Rating of 0 

SUBJECT 4: Rating of 0 

SUBJECT 5: Rating of 0 

SUBJECT 5: Rating of 0 

SUBJECT 6: Rating of 3 

SUBJECT 6: Rating of 4 

SUBJECT 7: Rating of 3 

SUBJECT 7: Rating of 3 

SUBJECT 8: Rating of 2 

SUBJECT 8: Rating of 2 

SUBJECT 9: Rating of 0 

SUBJECT 9: Rating of 0 

SUBJECT 10: Rating of 5 

SUBJECT 10: Rating of 5 

SUBJECT 11: Rating of 3 

SUBJECT 11: Rating of 3 

SUBJECT 12: Rating of 5 

SUBJECT 12: Rating of 5 

SUBJECT 13: Rating of 5 

SUBJECT 13: Rating of 5 

SUBJECT 14: Rating of 2 

SUBJECT 14: Rating of 2 

SUBJECT 15: Rating of 5 

SUBJECT 15: Rating of 5 

SUBJECT 16: Rating of 0 

SUBJECT 16: Rating of 0 


Please provide your reason for the rating provided below: 

SUBJECT 1: Everything I was seeing in the PFD display was in the other displays. However, the PFD gave 
me better SA and honestly I didn ’t even look at the vertical display. 

SUBJECT 2: It certainly doesn ’t hurt to have that information available to you. 

SUBJECT 3: Not really needed. 

SUBJECT 4: VSD can give you a different perspective and perhaps it easier to quickly discern but since SVS 
was so good I didn y t really even use or need to use it. 

SUBJECT 5: Wasn ’t part of my scan when I had SVS. 

SUBJECT 6: The reason is obvious when you can peak out and see terrain the VSD doesn ’t have as much 
impact on SA. The simulated vision is a much accurate depiction of what is actually happening. 

SUBJECT 7: There is so much additional information in the SVS that the VSD doesn ’t add much 

SUBJECT 8: Well, the information adds a little so I gave it a 2 but in reality I probably could get all the 
information I needed with the Size X display. 

SUBJECT 9: You know, there just wasn ’t any useful information being given in the display as what I was 
seeing in the ND and synthetic PFD 

SUBJECT 10: Cross-check for the TAWS and the VSD. Nice to have the terrain. 

SUBJECT 11: Great cross-check. Without the VSD, however, I would have been as confident 

SUBJECT 12: You have terrain information with SVS and TA WS but couldn ’t hurt to have more terrain 
information 

SUBJECT 13: Good instrument to cross-check but SVS gave you what you need and it is an excellent display 
at it is. Liked the VSD but would pay more for SVS. 

SUBJECT 14: SVS captures everything that the VSD. The VSD gave a little easier look ahead as to how the 
terrain gradient looks. The VSD gives you the distance to terrain better than SVS where it is a little 
difficult to tell how far the terrain. 

SUBJECT 15: Time to go With the SVS being 3D, the VSD gives the 4 th dimension of time. This is what is 
coming next and this is how much time to respond to what the symbology is telling me or that I am ok. 

SUBJECT 16: Didn ’t even use it. No help. 
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22. Please rate how much increased situation awareness the vertical situation display provided during your 
approach to rwy. 7 using the SVS Size A. 


Approach to rwy 7 using the SVS Size A 
SUBJECT 1: Rating of 0 
SUBJECT 2: Rating of 4 
SUBJECT 3: Rating of 3 
SUBJECT 4: Rating of 3 
SUBJECT 5: Rating of 2 
SUBJECT 6: Rating of 4 
SUBJECT 7: Rating of 3 
SUBJECT 8: Rating of 3 
SUBJECT 9: Rating of 0 
SUBJECT 10: Rating of 6 
SUBJECT 11: Rating of 6 
SUBJECT 12: Rating of 6 
SUBJECT 13: Rating of 6 
SUBJECT 13: Rating of 2 
SUBJECT 14: Rating of 2 
SUBJECT 15: Rating of 5 
SUBJECT 16: Rating of 2 


Missed Approach rwv. 07 using Size A 


SUBJECT 1: Rating of 0 
SUBJECT 2: Rating of 4 
SUBJECT 3: Rating of 3 
SUBJECT 4: Rating of 3 
SUBJECT 5: Rating of 2 
SUBJECT 6: Rating of 5 
SUBJECT 7: Rating of 5 
SUBJECT 8: Rating of 3 
SUBJECT 9: Rating of 0 


SUBJECT 10: Rating of 6 
SUBJECT 11: Rating of 6 
SUBJECT 12: Rating of 5 
SUBJECT 13: Rating of 6 
SUBJECT 14: Rating of 2 
SUBJECT 15: Rating of 5 
SUBJECT 16: Rating of 3 


Please provide your reason for the rating provided below: 


SUBJECT 1: 1 would say the same reason as for Size X. Just didn ’t need it. 

SUBJECT 2: Well, it did help with the smaller display especially when 1 was in 60 FOV. Hard to pick out the 
terrain in the Size A with large FOV 

SUBJECT 3: Because the size A doesn ’t have integrated symbology, it was easier to move the VSD in to scan 
and I found that it was helpful because of the small FOV of the Size A 

SUBJECT 4: FOV of Size A is so small and minified with 60 FOV that the VSD did supplement the 
information presentation on SVS instrument that I didn ’t need with Size X 

SUBJECT 5: Size X was larger and I could see more terrain. It wasn ’t as minified as I saw with Size A. I 
wanted to leave Size A in 30 on miss but the VV kept getting pegged. But when I went to 60, the scene was 
less useful on size A whereas it was still ok on Size X. Therefore, the TA WS and VSD became more 
valuable to me. 

SUBJECT 6: Smaller size has less SA with just SVS so VSD helped a little more than X 

SUBJECT 7: If both pilots are going to have it, the PNF can use it to reference during miss 

SUBJECT 8: Same reason but Size A is not as good for SA so having the VSD would be nice to have. 

SUBJECT 9: Again, no information that 1 couldn 7 ’ get from the other displays. I didn ’t even look at it. 

SUBJECT 10: The terrain was so compressed and hard to see; cluttered. 

SUBJECT 11: SA for terrain was excellent and the VSD provided additional check on terrain clearance. 

SUBJECT 12: Same 

SUBJECT 13: More advantageous since you don ’t ’ get a strong sense of terrain awareness that you get with 
Size X 

SUBJECT 14: Same as Size X 

SUBJECT 15: Same 

SUBJECT 16: Didn ’t really use it all. Some cross check but I can see how it could be somewhat helpful but 
SVS is so much better than it really isn ’t useful. 
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23. On a scale of 0 to 100, please indicate your assessment of the % added by the VSD in terms of safety 
improvement for the following display concepts: 

EADI w/ TAWS: 

SUBJECT 1: 35% 

SUBJECT 2: 50% 

SUBJECT 3: 40% 

SUBJECT 4: 50% 

SUBJECT 5: 50% 

SUBJECT 6: 40%o 
SUBJECT 7: 30% 

SUBJECT 8: 35% 

SUBJECT 9: 0% 

SUBJECT 10: 70%> 

SUBJECT 11: 60% 

SUBJECT 12: 60% 

SUBJECT 13: 70%o 
SUBJECT 14: 70%o 
SUBJECT 15: 70%o 
SUBJECT 16: 50% 

SIZE X: 

SUBJECT 1: 10% 

SUBJECT 2: 30% (doesn ’t hurt to have I guess) 

SUBJECT 3: 10% 

SUBJECT 4: 10% 

SUBJECT 5: 10% 

SUBJECT 6: 20%> 

SUBJECT 7: 10% 

SUBJECT 8: 15% 

SUBJECT 9: 0% 

SUBJECT 10: 0% 

SUBJECT 11: 10% maybe 
SUBJECT 12: 15% 

SUBJECT 13: 20%> 

SUBJECT 14: 10% 

SUBJECT 15: 20%> 

SUBJECT 16: 0% 

SIZE A: 

SUBJECT 1: 10% 

SUBJECT 2: 30% 

SUBJECT 3: 20%o 
SUBJECT 4: 20%o 
SUBJECT 5: 25%o 
SUBJECT 6: 25%> 

SUBJECT 7: 15% 

SUBJECT 8: 15% 

SUBJECT 9: 0% 

SUBJECT 10: 60% 

SUBJECT 11: 15% 

SUBJECT 12: 20%o 
SUBJECT 13: 30% 

SUBJECT 14: 20%o 
SUBJECT 15: 20%o 
SUBJECT 16: 0% 
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24. If the CFIT was not detected, what was lacking in situation awareness? 

SUBJECT 1: Didn ’t detect and didn ’t think SA was lacking. Just saw bright red like every other run and 
assumed all was ok. He did see RA come on, but it was too late then. 

SUBJECT 2: N/A detected CFIT 
SUBJECT 3: N/A 

SUBJECT 4: No detection. TA WS didn I change and wasn I using the VSD as much as I should have 

SUBJECT 5: N/A 

SUBJECT 6: N/A 

SUBJECT 7: N/A 

SUBJECT 8: N/A 

SUBJECT 9: N/A 

SUBJECT 10: N/A 

SUBJECT 11: N/A 

SUBJECT 12: N/A 

SUBJECT 13: N/A 

SUBJECT 14: N/A 

SUBJECT 15: N/A; magenta line shifted over to the left about .5 mile. 

SUBJECT 16: Did notice it until the very end. 

25. At what point in the experimental run, did you notice a problem and possible CFIT encounter (i.e., position, 
time, event, etc.)? 

SUBJECT 1: N/A 
SUBJECT 2: 20-30 seconds 

SUBJECT 3: Hard to say. I definitely knew something was wrong about 1/ 2 way through the miss 
SUBJECT 4: N/A 

SUBJECT 5: About 2 miles before I went through the notch in the terrain 

SUBJECT 6: 9.9 DME; had intersected the 059 radial and then kept drifting further and further left so knew 
something was wrong 

SUBJECT 7: 10.4 DME stated something was wrong. The death dot went below the mountain range and 
then knew without really having to look at other information. 

SUBJECT 8: After making the turn to the left and nothing seemed abnormal but on the right turn saw 
velocity vector on terrain and then looked at VSD and knew something was wrong. After noticing hey I 
wasn I going to miss terrain, initiated the standard recovery maneuver (target is 15 pitch). 

SUBJECT 9: That was strictly due to the photo terrain display. 

SUBJECT 10: 5 nm from terrain could see the commanded flight director was taking me into the terrain. 
SUBJECT 11: 6 nm from go around I could tell 
SUBJECT 12: 8.2 DME 

SUBJECT 13: 11.4 DME but wasn ’t really sure that there was a CFIT but knew something was up and 
normally wouldn ’t have flown to such red in ANY OF THE RUNS. So, although he said something just 
before CFIT, he was unaware that there was actually a potential of a CFIT but rather knew something was 
a little different but not sure “ exactly what I was feeling” A gut reaction. 

SUBJECT 14: 10.8 DME I knew something is wrong. The VSD gave me the best information but without 
SVS I really couldn ’t be sure. I got the feeling much earlier than when I said so but SVS I couldn ’t tell how 
far but I knew something was wrong. The VSD confirmed it for me. 

SUBJECT 15: Around the last right turn. 

SUBJECT 16: N/A 
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26. Which instrument alerted you to the possible CFIT? 

SUBJECT 1: N/A 

SUBJECT 2: SVS — noticed was left of course then I looked at VSD to confirm and knew it for sure when I 
saw the orange wedge 

SUBJECT 3: SVS without a doubt. VSD helped to confirm 
SUBJECT 4: N/A 
SUBJECT 5: SVS 

SUBJECT 6: SVS definitely without a doubt; RMI was last and the 1 degree it was impossible to tell that it 
was different 

SUBJECT 7: SVS no doubt about it. VSD was nice to cross-check 

SUBJECT 8: TA WS first alerted something was not right. Then SVS of course to check and saw velocity 
vector was on terrain that told me that this wasn ’t the usual red I have seen on the last 21 runs. Couldn ’t 
see the valley on the HUD so my course guidance must be off. 

SUBJECT 9: SVS definitely. 

SUBJECT 10: SVS first, then shortly thereafter the TAWS. Problem with TAWS is that it is always bright red 
but you could see with the trace (noodle) that it was on a path to terrain. 

SUBJECT 11: VSD but the SVS told me what to do gave me an option. 

SUBJECT 12: SVS; I couldn ’t see that the notch was way off to the right. TA WS was last but VSD really 
helped in combination to confirm SVS display 

SUBJECT 13: TA WS. VSD didn ’t come into decision making instead was focused on TA WS 
SUBJECT 14: SVS first but VSD confirmed it for me. 

SUBJECT 15: SVS first with the cross-check with TAWS and VSD 
SUBJECT 16: N/A 

27. Did you feel you had enough information to safely avoid the terrain? a. Prior to first noticing the CFIT? 

SUBJECT 1: N/A 

SUBJECT 2: Went above terrain at about 410 feet and avoided. Felt was a little late in seeing terrain and 
avoiding but could do so. 

SUBJECT 3: The SVS display lets you see the terrain well before it becomes a danger . However, it was a 
little hard to determine how far but with repeated times flying it, I was in tune with the distance 
SUBJECT 4: N/A 

SUBJECT 5: Without SVS there would have been very little change of me being able to discover that 
something was up. However, because SVS gives you such an intuitive image you can ’t help but notice that 
something was wrong. Forget the RMI it was no help and no one uses it anyway. VSD could be useful but 
it will require a lot of training. I noticed that once I came close the VSD started giving me an indication 
that everything was ok again. Not very good presentation of dangerous terrain. It should highlight, beep, 
.lightup.... something instead of showing some yellow 5 miles away and then the peak disappears when you 
come up close to the terrain. 

SUBJECT 6: Yes. 

SUBJECT 7: Yes 
SUBJECT 8: Yes. 

SUBJECT 9: Yes. With a photo HUD I may have caught it sooner but the RA and photo depiction let me 
know 

SUBJECT 10: The SVS display gave excellent terrain information which allowed me to manage my path and 
energy to safety avoid the terrain 

SUBJECT 11: VSD alerted and the SVS gave the guidance on what to do that the VSD doesn ’t 
SUBJECT 12: Early alert and avoided the terrain. 

SUBJECT 13: No. Honestly, I would have turned right but given the distance to contact it would have been 
possible but not sure. 

SUBJECT 14: Yes. 

SUBJECT 15: Felt I could get over the peak so I didn ’t need to laterally miss the terrain since I knew that I 
could fly over it and so I was sure that I was going to be ok. 

SUBJECT 16: N/A 
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28. Upon noticing a potential CFIT? 


SUBJECT 1: N/A 
SUBJECT 2: Yes 
SUBJECT 3: Yes. 

SUBJECT 4: N/A 
SUBJECT 5: Yes. 

SUBJECT 6: Yes; it was definitely the SVS and everything looked good but it was clear that something was 
wrong on the SVS display. I think that everyone who didn ’t get vision would hit the mountain without a 
doubt. I don ’t see how you could not hit it given how compelling and subtle the error is. 

SUBJECT 7: Yes. 

SUBJECT 8: Yes. After 20 in a row, it obviously looked wrong. 

SUBJECT 9: Yes. 

SUBJECT 10: Yes. 

SUBJECT 11: Yes 
SUBJECT 12: Yes 

SUBJECT 13: Again Yes and No given the difficult of discriminating versus other runs that were similar 
SUBJECT 14: Really appreciated the SVS compared to the baseline for avoiding the terrain. The VSD also 
was great and helped significantly. 

SUBJECT 15: I think so. 

SUBJECT 16: N/A 

29. What is your best estimate of the closest approach to terrain that you experienced during the last experimental 
run with the CFIT encounter? 

SUBJECT 1: N/A 

SUBJECT 2: 250 feet because of orange wedge indication 
SUBJECT 3: 200 feet 
SUBJECT 4: N/A 
SUBJECT 5: 5 00 feet 

SUBJECT 6: 1200 feet vertical; maybe .25 laterally 
SUBJECT 7: Within 200 feet but knew I was going to clear it. 

SUBJECT 8: Run stopped but I was 1000 ft RA 
SUBJECT 9: About 250 feet. 

SUBJECT 10: About 5 DME noticed the terrain. 

SUBJECT 11: Well above terrain when scenario stopped. 

SUBJECT 12: 1000 feet 

SUBJECT 13: 7500 feet and looked like I was right on top of the mountain. 

SUBJECT 14: 1000 feet laterally 
SUBJECT 15: Right around 300 feet vertically 
SUBJECT 16: N/A 
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30. What were the disadvantages / advantageous of the display condition for the CFIT detection? 

SUBJECT 1: N/A 

SUBJECT 2: Would really like to have an indication on PFD showing MSA (i.e., a number on a peak) 

SUBJECT 3: SVS with the VV lets me know well ahead of the terrain that if I continue on the path I am on I 
was going to hit the mountain; no disadvantages except that it would be nice to be able to tell how high the 
mountain is 

SUBJECT 4: N/A 

SUBJECT 5: Excellent display. I would suggest that some indication needs to be in the PFD to help make the 
connection with the ND. I found myself looking at the EGPWS and seeing red but not really sure exactly 
what peaks were what. Sure, I knew that the green wedge was telling me what my FOV on the PFD but it 
did take some extra mental steps. Also, having MSA, terrain elevation information, etc. would help. Maybe 
a combination of generic and photo would be best to give the best of what both have to offer 

SUBJECT 6: Pilots don ’t fly into mountains they see 

SUBJECT 7: It was good. The fact that the velocity vector was indicating the path and the photo texture was 
well enough defined that I recognized that the flight path. Enough information to make acquarate 
assessment. 

SUBJECT 8: Having the terrain was a big advantage. That allowed me to at least process hey something is 
not right and to avoid all I had to do was put the velocity vector. The TA WS and VSD tells me something is 
wrong but the SVS can cross check and in addition and very significant give me the escape guidance I need 
to avoid terrain. 

SUBJECT 9: Huge advantage to CFIT detection and it certainly avoided what would have been an accident. 

SUBJECT 10: It showed you the real world picture of where the aircraft was headed even though you 
couldn ’t see outside. It was excellent and I really got high SA using the display. 

SUBJECT 11: The real positive is if something is not right it gives you the information you need to remain 
safe. If you got the baseline and everything is working just fine, but as soon as something goes wrong SVS 
really pays off. 

SUBJECT 12: flight director was smaller and less terrain information but there was enough information to 
discover that the path was incorrect. Size X would have been better but Size A was adequate. 

SUBJECT 13: Huge advantage to CFIT detection and it certainly avoided what would have definitely been 
an accident... the SVS allowed me to see the picture of what was happening that the VSD and TAWS was 
telling me. Pull up, stupid yet without the SVS it would have been hard to say “yes, I’m off path and I am 
going to go into the mountain A picture may be worth a thousand words, but with synthetic vision it is 
more like it is worth a thousand lives! 

SUBJECT 14: The SVS allowed me to see the picture of what was happening that the VSD and TA WS was 
telling me. Pull up, stupid yet without the SVS it would have been hard to say yes, Pm off path and I am 
going to go into the mountain. A picture is worth a thousand words, or better, a thousand lives!! 

SUBJECT 15: Liked that you could correlated MSAs on SVS with Jeppesen. On GA mode, putting MSA on 
display would be very helpful. Combined with a EGPWS, it would be a complete package and no 
disadvantageous. 

SUBJECT 16: N/A 
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Appendix G: Pilot Responses to Post experiment Questionnaire 


1. Please discuss which display concept that you experienced which provided the most enhanced situation 
awareness in making the missed approach at EGE? 

SUBJECT 1: The Photo Realistic View by far provided the greatest margin of safety. In fact was solely 
responsible for completing the maneuver safely. 

SUBJECT 2: Size X with photo realistic. Size matters. It was easier than Size A and the color was better 
than the HUD. 

SUBJECT 3: HUD - with the ground depiction allowed for ease of transition for scan. 

SUBJECT 4: The Size X with photo realistic terrain provided the most awareness since I could expand field 
of view with a high pitch attitude. 

SUBJECT 5: Overall, I would say the X size Photo realistic. The HUD might work well also ie when you 
break out on the approach or when ever you see the real terrain. 

SUBJECT 6: Both the HUD and HDD were great when presented with terrain info. The terrain info, even a 
small bit of info was helpful. 

SUBJECT 7: The ( HUD Photo ’’provided me the best SA because of its 1 to 1 real and correctly positioned 
image. I would prefer color HUD image with enhancement capability (i.e. draw a runway outline that is 
enhanced) or enhance significant objects like towers or highest mountain peaks. 

SUBJECT 8: Size X. Better overall display for SA when used with vertical situation display. 

SUBJECT 9: In most cases I felt the X size display with photo realistic gave me greater SA. Sometimes 
(emphasis) the HUD display would momentarily be hidden (green on green). 

SUBJECT 10: The best presentation was the “Size X- photo realistic”. It provided the most realistic info in 
an easy, format of sufficient size. 

SUBJECT 11: NASA Size X HDD provided highest level of SA. The terrain portrayal and size were key SA 
builders. No real difference for photo or generic terrain. 

SUBJECT 12: Size X with photo realistic terrain 

SUBJECT 13: Size X photo realistic provided the most SA due it’s ability to replicate the “outside world” the 
best. 

SUBJECT 14: 1 preferred the Size X generic display. Selecting wider ranges when in close proximity to 
terrain gave a reassurance that lowered tension while flying toward rising terrain. 
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2. Please comment on the NASA PFD display concepts and what you particularly liked and thought were the most 
salient features for improving safety, performance and enhancing your situation awareness 

SUBJECT 1: The Size X Display combined with a photo realistic display were huge advantages. 

SUBJECT 2: Again, Size X Although a HUD in the later stages (Last 60 seconds before touchdown) would 
help too. 

SUBJECT 3: (no comments) 

SUBJECT 4: Photo realistic and generic terrain overlays were great for maintaining SA. VSD is a great 
enhancement that gives accurate distance of upcoming terrain. 

SUBJECT 5: The HUD and large PFD worked well in giving you a heads up when turns and descents were 
upcoming. 

SUBJECT 6: 1 liked the Size X concept the best in both HUD and HDD, it was more helpful and easier to 
comprehend data. The versions with photo realistic detail were the greatest asset. Generic data seemed 
poorest on SA. 

SUBJECT 7: The Size X was the best heads down, followed by Size “. A ”. I would actually prefer a 
combination generic view - when up and away from the ground (because definition is better than photo) 
and use photo real below a certain altitude say 1000 ’ or 500 ’. This would also alert you to “ You are 
getting close to the ground”. See #22 for continuation. 

SUBJECT 8: Size X with VSD package, tunnel for better “down stream ” SA, ghost airplane easier to follow 
than current 2 bar baseline system. 

SUBJECT 9: These display overall were outstanding. Easiest to use in a quick glance for terrain clearance 
was the VSD. HUD display was exceptional SA on last run where NA V system was “off”. 

SUBJECT 10: The best feature was the terrain info. Followed by the “flying triangle” flight director. 

SUBJCECT 11: The terrain portrayal was the key to improving SA and safety. The crow ’s feet tunnel was 
nice to have. 

SUBJECT 12: photo realistic terrain depiction on all of the different displays 

SUBJCECT 13: All the displays except the baseline provided good guidance, situational awareness through 
the terrain displays and intuitive feedback on the aircraft state. All the things necessary to complete a 
successful flight. Best features would be the photo realism, ghost a/c guidance with velocity vector. 

SUBJECT 14: Guidance/track projections superimposed on 3D depictions of terrain and cultural features 
increased situation awareness and comfort levels. Would greatly aid contact approaches in marginal 
conditions. 
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3. Please comment on the NASA HUD display concepts and what you particularly liked and thought were the 
most salient features for improving safety, performance and enhancing your situation awareness 

SUBJECT 1: 1 prefer the HUD for transitioning from instrument flying to flying by strictly visual references. 
As in transition to landing from an approach to minimums. Although this couldn ’t be proven in this 
particular experiment. 

SUBJECT 2: Any HUD is a tremendous improvement. The NASA HUD is well designed and well laid out. 

SUBJECT 3: Relative vertical positions is easily determined. 

SUBJECT 4: Terrain overlay on the HUD allows pilot to view the big picture while keeping eyes outside of 
aircraft. HUD allows for quicker crosscheck of aircraft performance compared to PFD. 

SUBJECT 5: Great out the window SA no need to look inside for info. Will work great in landing transition 
environment. 

SUBJECT 6: The terrain feature was great, and probably a greater asset at night when easier to look at and 
no terrain is visible. 

SUBJECT 7: The biggest enhancement is seeing where you are (FPV) and the real world image. Nothing is 
more compelling than seeing the mountain. People will fly into red (because of all the false warnings they 
have seen in years of flying - but they will never fly into a picture of a mountain). 

SUBJECT 8: All the good things of the PFD with a Heads Up and out of the cockpit for better traffic and 
outside world SA. 

SUBJECT 9: The tunnel depiction was very good for placement on the curved approach to final. Also ghost 
aircraft was helpful. 

SUBJECT 10: The HUD concept offers time critical info from the most useful location for the flying pilot. 

SUBJECT 11: The terrain portrayal was the key to improved SA and safety. 

SUBJECT 12: photo realistic terrain 

SUBJECT 13: The guidance program is good. The terrain provides additional situation awareness but 
doesn 7 have the impact of the photo realistic displayed on the PFD display. 

SUBJECT 14: HUD concept would be especially useful for retrofit into non-glass flight deck applications. 
Heads-up feature should be especially useful when flying/transitioning in and out of clouds and haze. 
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4. Please discuss any crew coordination or operational issues that you think may be a problem with using the SV 
displays (especially the HUD). 

SUBJECT 1: A single display HUD without adequate instrumentation on the FO’s (first officer's) side is 
contrary to the latest CRM Philosophy. 

SUBJECT 2: If captain (only) has HUD. FO will need something to be able to accurately back him up. 

Flight director & magenta line at a minimum. 

SUBJECT 3: Very little as far as problems 

SUBJECT 4: With terrain overlay, the HUD can become very crowded with information. However, each 
pilot and situation are different so I prefer as much info displayed as possible but have the option to de- 
clutter the HUD. 

SUBJECT 5: For those who have flown a HUD or used NUG ’s you must remember you are not looking at 
real-time terrain but generated terrain. 

SUBJECT 6: Depending upon night or day operation it may be best to offer one pilot terrain info on the 
HUD while the other pilot the same terrain info on the HDD. 

SUBJECT 7: It is obviously best if both crewmembers have a HUD. A repeater heads down is second best 
and no HUD or repeater for second pilot is of little value. 

SUBJECT 8: Because of increased ability to do things that were beyond a comfort level in the past - more 
intra-crew communication must exist to alleviate questions like “ Where are you going? ” or “What are you 
doing? ” 

SUBJECT 9: Crew coordination problems WILL occur if only 1 pilot has a HUD. And without a good 
battery of other items (VSD & ND with generic or photo realistic) 

SUBJECT 10: Airline operations require the flying pilot to go heads up on final. An effective system would 
therefore require both HUD and HDD for both pilots to have the same info at “Low levels ” 

SUBJECT 11: Not all info required is on the HUD and to get all required information inside instruments 
needed to be checked. 

SUBJECT 12: Unless the HUD is installed at both pilot stations thus no-terrain reference backup if one fails. 
Switching FOV or other features will add to workload of pilot not flying who is already involved in other 
tasks such as setting up fit data or his side & ATC communications. 

SUBJECT 13: Operationally the need to change the FOV during the go around is a problem. The last thing a 
pilot needs is to lose the guidance cues or have a difficult time seeing them during a high workload task. 
With the HUD, if only one is on the flight deck (as done on today ’s ops) it is imperative the other pilot has 
ALL the same cues and guidance as the pilot with the HUD. 

SUBJECT 14: With a crew application, it would be important to ensure both crew members were referencing 
the same scale as guidance appeared different at different ranges. With HUD, might want to have on with 
and on without HUD. 


94 



5. Please comment on the “opaqueness” of the NASA HUD concept. What are your opinions on its operational 
safety and benefits of having a synthetic terrain on the HUD? Please provide your opinion with the 
acknowledgement that you may have only seen the HUD in the simulator. 

SUBJECT 1: Can you put a photo realistic display on a HUD? The HUD presents limitations in contouring I 
assume is due to the “see through ” requirement. Without further testing my opinion is Size X with photo 
realistic (PR) would be preferred for general use over the HUD. 

SUBJECT 2: This is an issue. Must have a method to transition to outside - unless this is planned/intended 
to go into a zero-zero field. 

SUBJECT 3: (no comments) 

SUBJECT 4: Having terrain display in the HUD is an enhancement even though it is opaque. It provides a 
crucial element of pilot awareness to his environment. It is good enough for airline/commercial aircraft. 

SUBJECT 5: Good SA enhancer. Need to be able to have the synthetic terrain go away when you can 
actually see the ground. 

SUBJECT 6: I’ve used the HUD many times before and find it an awesome asset and the terrain feature is a 
definite plus. Perhaps the ability to use or not use it depending upon the situation is my only operational 
suggestion. 

SUBJECT 7: 1 have flown the F-16 LATIRN (IR image) HUD at night. If it is adjusted correctly it is no 
problem, you see the projected image over the top of the real world and when real world is viewable you 
see real world. 

SUBJECT 8: While Photo would be the preferred display - even generic provides enough detail to provide 
better SA. Would feel better about critiquing the HUD after seeing how it looks (in the) real world in VMC 
conditions - It may be too much to look at if operating conducted with ground contact visually. 

SUBJECT 9: 1 used a HUD in the USAF for nearly 20 years. The NASA HUD was exceptional for SA. Some 
minor changes may be needed to add contrast and/or warning messages to be totally head's out. 

SUBJECT 10: Opaqueness could be a problem without contrast and brightness (terrain) controls. 

SUBJECT 11: Could be a problem, but not likely. Should be implemented so it could be moved out of the 
way during VFR conditions possibly. 

SUBJECT 12: Having the terrain depicted on the HUD is great when in IMC. In VMC being able to easily 
declutter and adjust intensity of HUD data is essential. Photocell and adjust intensity automatically with 
manual backup. Declutter for terrain & data needs to be easy for pilot flying to do himself. Confused 
communications about display between crew could be dangerous. 

SUBJECT 13 .‘Opaqueness of the HUD is good. The terrain is a good addition but I saw little benefit with 
any terrain display other than a generic presentation. I did have some confusion with the display during 
turns caused by the tunnel corner “ ties ” interfering with the guidance cues and “pitch ladder ”. 

SUBJECT 14: 1 think the configuration should allow quick/easy look around or turn-off if being used during 
approach phases. 
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6. Please discuss the use of the tunnel to perform the missed approach to rwy 7. What features of the tunnel did 
you particularly find most helpful in maintaining situation awareness? 

SUBJECT 1: 1 see no advantage to the tunnel Especially in Size A it was more a distraction than help. 

SUBJECT 2: Nice piece of additional cues to give pilot indication of proper path. 

SUBJECT 3: (no comments) 

SUBJECT 4: The tunnel was an enhancement because it provided a few second “ look ahead ” of what change 
in course was about to happen. It allowed pilot to anticipate any turns coming up. 

SUBJECT 5: Gave you a forecast of where you wanted to make A/C go. 

SUBJECT 6: The tunnel concept was great! But only used it on the approach and circle not the missed appr. 
The MAP seems easier with only a flight director though. 

SUBJECT 7: The tunnel is a nice big picture but I would prefer it to be more dynamic (i.e. lines get longer as 
you pass by them.) 

SUBJECT 8: Better SA for flight path (descents, climbs, turns) 5-10+ seconds down the road. 

SUBJECT 9: 1 didn ’t really see much tunnel on missed approach. It was very beneficial during the approach 
to rwy 7. 

SUBJECT 10: Ineffective. Once the tunnel was allowed to pass out of view there was no flight guidance. It 
would be very helpful to have traditional cues on the edges of the PFD. 

SUBJECT 11: On the missed approach I did not really use the tunnel. I followed “flight director ” and cross 
checked other instruments. 

SUBJECT 12: Didn ’t really see the tunnel on missed approach it was most helpful as a turn cue to anticipate 
movement of ghost airplane. 

SUBJECT 13: The tunnel provides good course guidance when in the tunnel. Outside the tunnel during a 
turn it is initially difficult to (illegible) the lateral and vertical deviations and the correct direction to 
return to the tunnel Staying in the tunnel with guidance proved all the necessary SA needed to complete 
the approach. 

SUBJECT 14: Flying in the tunnel gave comfort in providing a “zone” showing reasonable boundaries. This 
way one doesn ’t have to over-concentrate on flying some exact line through the sky. Also was useful in 
predicting flight path & providing a sense of timing during flight. 
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7. Was there any symbology on the NASA HUD or HDD display concepts that you found confusing or difficult to 
use? Any symbology cues that you found helpful? 

Symbology that was confusing: 

SUBJECT 1: The envelope arrows could be confusing. Especially on Size A. Too close to other useful info 
with the same color. Too hard to differentiate. 

SUBJECT 2: None. 

SUBJECT 3: On Size A - decluttering would be appreciated 

SUBJECT 4: During the missed approach, the flight director changed from ghost airplane to just a ball. This 
transition was mildly confusing. 

SUBJECT 5: The flight director airplane. Felt it was too big and instead of following it, I wanted to fly form 
on it. 

SUBJECT 6: The FD wasn *t helpful on missed appr in circle only mode. 

SUBJECT 7: Water line cue is not needed and did confuse me on one run. 

SUBJECT 8: Heading cue needs to be larger. Noodle (or turn trend) seems overly sensitive. 

SUBJECT 9: need a velocity vector on missed approach that is easier to see/follow. 

SUBJECT 10: tunnel - difficult 

SUBJECT 11: Size A HDD unity FOV difficult to see symbology; Baseline with 60 degree FOV symbology 
does not do well. In general trying to fly ‘ ball in ball ’ will cause PIO ’s 

SUBJECT 12: Dual cue flight director. A single cue FD is much easier to interpret and does the same job. 

No need to make things any harder. 

SUBJECT 13: Heading numbers compressed. Sometimes difficult to read ghost a/c over correcting in turns 
increased workload some. Vertical speed display needs to be moved out of altitude tape on the SVS PFD 
displays. Tunnel corners conflict wit pitch ladder during turns on HUD display. Radar altitude display 
over altitude tape on some displays difficult to see. 

SUBJECT 14: Lack of pitch reference at the longitudinal axis on the HUD (attributed to a software problem). 
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Symbology that was helpful: 

SUBJECT 1: The magenta path triangle was a useful trend info, especially in Size X. 

SUBJECT 2: Ghost airplane was a great tool 

SUBJECT 3: Great gates on HDD - thought this allowed for rapid recognition of special orientation. 

SUBJECT 4: Ghost airplane was helpful because during turns it provided two-dimensional info. 

SUBJECT 5: Tunnel, speed symbol. 

SUBJECT 6: The phantom airplane and tunnel concept were great. 

SUBJECT 7: Flight Path Vector (FPV) - Worm - energy carat great. Radar altitude center box should pop 
up at 1000 ’ - also radar altitude thermometer scales are excellent (drawn radar alt thermometer drawn). 
Scale goes down closer you get to ground. This scale would pop up on right of display when below 1000 ’ 
replacing or next to altitude scale. 

SUBJECT 8: Tunnel, ghost A/C, VSD 

SUBJECT 9: ghost ACFT & tunnel; terrain (generic, photo & HUD) 

SUBJECT 10: “flying triangle” flight director 

SUBJECT 11: Tunnel on HDD for approach, turning symbology on follow me a/c, yellow terrain bar that 
comes into view when w/in 200 ft AGL. 

SUBJECT 12: All of the symbology was helpful but t was almost too much info to scan interpret and then use 
at one time. Relevant info needs to appear and disappear automatically based on phase of flight (i.e. no 
need for radar alt at cruise above 1500) 

SUBJECT 13: Ghost aircraft and velocity vector. Speed trend bar and speed cue attached to guidance circle. 

SUBJECT 14: “Tunnel” , follow-me aircraft symbol, god’s eye view of programmed ground track. 
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8. Please provide any comments regarding whether field-of-view (FOV) requirements may change as a function of 
phase of flight (Example: “I would like to use 90 FOV for cruise but prefer unity FOV for approach:). 

SUBJECT 1: 60 degrees was preferred until close in (within 3 nmi and 1000’ above), then 30 degrees seemed 
to be a better depiction - more to scale so to speak. 

SUBJECT 2: 1 found 30 ideal for most phases. 60-90 worded OK in hard turns so you could see where 
you ’re going. Unity seemed overly restrictive (who would fly a real airplane if the entire front windscreen 
was only 8”x8” window?) 

SUBJECT 3: 1 would like to use 90-60 at altitude. On climb and descent scan thru 30. Unity would be good 
in a short to in close. Unity is least used. 

SUBJECT 4: Each pilot has a different preference depending on situation. I preferred 30 FOV for approach 
and 60 FOV for missed approach. 

SUBJECT 5: It was easier to fly in larger field of view. Unity was too sensitive and in landing config picture 
dropped off the bottom. 

SUBJECT 6: Me personally, I liked 30 degree best for all phases of fight, but I guess it was cool being able 
to change this if I wanted to. 

SUBJECT 7: The FOV problem was actually an elevation problem. You had to change FOV to see more 
below the FPV. It would be best I think to have a set FOV and lower the viewable area below the FPV. 

SUBJECT 8: Did not like unity for any phase of fight. Felt 60 degree FOV provided best all phase of fight 
integration, however symbology can become cluttered at times with a resultant situation where ghost 
aircraft target dot becomes hidden. 30 degree FOV symbology /clutter seems most usable, however 
correction sensitivities seem magnified leading to PIO or mild over controlling of ft. Ctls. Preferred 90 
degree FOV for ‘ low clearance ’ ridge crossing because of detail however, corrections and sensitivities are 
dampened when they may be needed most. 

SUBJECT 9: 60 degree FOV for cruise though not as precise, 30 degree FOV for final, unity was not useful 
to me 

SUBJECT 10: It was possible to select a FOV that was too narrow to encompass the missed approach 
procedure & course guidance. 

SUBJECT 11: Yes, the requirement will change, unity would be nice to use all the time ... like the real world. 
The further above ground & away from the airport the wider the FOV I would use. 

SUBJECT 12: For Size A display the 30 degree FOV seemed better for approach but the 60 degree FOV gave 
better pitch reference ladder bars for missed approach. Pilot flying shouldn ’t be thinking about changing 
FOV at missed approach point. Change should be automatic. 

SUBJECT 13: FOV was changed from 30 to 60 during the go around. The rest of the time I used the 30. The 
change was made to be able to see the guidance better. 

SUBJECT 14: 1 found myself using the 30 degree FOV for pre-approach maneuvering. 60 degrees during 
turning descent, 30 during initial climout. As high terrain approached 60 & then 90 revealed close high 
terrain. 
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9. Would you prefer that the FOV settings be pilot- selectable or engineered into the system (i.e., FOV changes 
remain static based on phase of flight)? 

SUBJECT 1: Pilot selectable. 

SUBJECT 2 .‘Pilot selectable! 

SUBJECT 3: (no comment) 

SUBJECT 4: Pilot- selectable! Everyone has a different preference however the only exception would be 
during a missed approach. I found it mildly difficult to change FOV while executing a missed approach - 
This is a crew coordination item where the pilot not flying changes FOV when commanded by the pilot 

flying. 

SUBJECT 5: Pilot select. 

SUBJECT 6: Absolutely pilot selectable! Everyone has a different 

SUBJECT 7: Pilot selectable 

SUBJECT 8: FOV changes should NOT remain static but would like to see a less cumbersome way to control 
FOV & range, eg. Making it a ‘HOT AS’ (F-18 option) system where changes can be made via a 
pushbutton on either the stick or throttle (actually mouse operation can also be conducted this way via an 
index finger mouse/push button on stick or throttle). 

SUBJECT 9: Pilot selectable since different approaches & terrain may require a system no “ hard wired” 

SUBJECT 10: Both. Selectable for all operations with an “auto ” feature for the most advantageous FOV to 
“ pop up ” during emergency and abnormal OPS including missed approach procedures. 

SUBJECT 11: Pilot selectable but should allow pilot to set automatic changes such as going 60 degree FOV 
on missed approach. 

SUBJECT 12: Engineer changes that would switch for optimum clarity automatically for the phase of fight 
with pilot override for failures or emergencies. 

SUBJECT 13: Both. Default changes based on phase of fight (and studies determining the best FOV per 
flight phase) but allow pilot selectable option due to pilot preference or requirements. 

SUBJECT 14: Pilot selectable! I found the trackball hard to use an would have preferred a rotary switch for 
the FOV selections - This also could be a crew coordination item (task non flying pilot to make the 
changes). 


100 



10. If you had to select between two different FOVs that may be pilot selectable, which two FOVs would be chosen 
and why? 

SUBJECT 1: 60 degrees and 30 degrees. The only 2 that seemed comfortable. Those 2 seemed to more 
represent actual conditions. 

SUBJECT 2: Depends on situation - 1 would choose 30 & 60. 30 for most situations; 60 - during turns, 
increases SA, during missed approach, too. 

SUBJECT 3: 90-30; 90 at altitude and 30 on T/O and arrival. 

SUBJECT 4: Unity and 60 degree keeps a “normal” picture while 60 degrees provides adequate FOV for a 
missed approach. 

SUBJECT 5: 30 & 90; 90 was good for approach phase; 30 was needed for landing phase. 

SUBJECT 6: 30 degrees was my preference. Easier for me to use and view. 

SUBJECT 7: Unity and 45 degrees. 

SUBJECT 8: 30/60 - explained in answer #8 

SUBJECT 9: 30 & 60. See #8 

SUBJECT 10: 30 degree approach - easiest scale to use. 90 degree arrival & missed approach - offers a 
wide FOV for terrain info. 

SUBJECT 11: For Size X- unity and 60 degree for Size A - 30 degree and 60 degree. I think 90 degree is 
not required and unity is difficult to use on Size A. 

SUBJECT 12: 30 degrees or 60 degrees. 30 seemed detailed enough for approach. 60 was good for missed 
approach. 90 is a little too wide for good detail resolution of man made objects and airports. 

SUBJECT 13: During runs only FOVs I used were 30 and 60. Tied all the views but these were the only 2 
used. 

SUBJECT 14: If I could only have 2 FOV’s to use, Ed choose 30 and 60. Unity seemed too much like looking 
through a straw. 90 could be left out as 60 provided expanded view; at 90, you ’ re essentially past the 
problem - terrain. 
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1 1 . Given a choice about where to display the image, what would you recommend for: 


Subject Recommendation for Type of Display for SVS 



Display Type 


Please comment on your choice: 

SUBJECT 1: Choices without cost factors. With cost factors I would choose Size X Photo slightly above 
HUD. 

SUBJECT 2: Of course, more is better. A HUD would help for the critical transition to outside visual 
references, but it is somewhat harder to fly during heavy maneuvering due to its limited FOV. 

SUBJECT 3: HUD is great for safety as to other aircraft. Just feel both allow for both pilot flying and non- 
flying pilot situational awareness. One peek is worth a thousand scans. 

SUBJECT 4: Cost aside, I am big proponent for HUDs on commercial aircraft - keep eyes outside aircraft as 
much as possible. 

SUBJECT 5: Seems the refit for a HUD on 757 would be cost prohibitive if money were no issue then both 
would be great. 

SUBJECT 6: As long as its pilot selectable why not offer this info to both locations and let the plot decide 
where he/she wants to view it. 

SUBJECT 7: Head up is far superior because it gives you one for one with the outside world and reduces 
transition to outside world when landing. Head down is good backup. And helps those not HUD pilot 
transition easier. 

SUBJECT 8: (no comment) 

SUBJECT 9: Unless HUD is available to both pilots, crew coordination would necessitate both systems as 
backup. Also consider target audience and their experience with or w/o a HUD. Chances are initially 
most will feel comfortable w/ some sort of head-down system needed as a backup. 

SUBJECT 10: Most useful for the pilot to have the same information available when going from heads down 
to heads up during the final landing phase as well as all other operations near and around terrain. 

SUBJECT 11: The display is for flying in IFR, so it is natural to be inside (HDD) putting info on HUD could 
cause problems and is not as good (two colors) as the HDD. 

SUBJECT 12: Unless the HUD is installed at both pilot stations the Head Down display makes more sense. 
Head down is part of the already existent instrument scan, would probably require less training time and 
achieve the same goal. 

SUBJECT 13: Use of both head up/down would allow non flying pilot the ability to better monitor the 
aircraft path when the flying pilot is using a head up display. 

SUBJECT 14: For Airbus - PFD (ie size X) would be enough - easy to see the detail. For 757 I’d make a 
HUD optional -for future acft the same. I frankly think that the low frequency of fights into poor vis and 
challenging terrain might not justify expense of a HUD just for this; however if the aircraft is going to have 
a HUD, this is a great option. 
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12. Please rate your estimate of the lowest acceptability of the following display sizes and image resolution based 
on using the display to fly the airplane. 


Subject rating of lowest level of terrain texturing 
acceptable for cruise flight 



Size A Size X HUD 


■ Photo 

□ Generic 

□ Neither 


Display Type 


Subject rating of lowest level of terrain texturing 
acceptable for approach and landing 
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Size A Size X HUD 

Display Type 


Subject rating of lowest level of terrain texturing 
acceptable for a missed approach 
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Size A 
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I Photo 

□ Generic 

□ Neither 


Size X 

Display Type 


HUD 
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13. In the figure, please draw or indicate the changes you would want to incorporate into a head-up SVS display or 
any changes that you think would enhance the performance and safety benefits of the SVS HUD. You may 
draw directly on the figure or use the box below. 



SUBJECT 1: 1 would like to see a wind arrow or wind directin & speed in the top right(or left) of the screen. 

SUBJECT 2: No complaints. Nice depiction. Works for me. 

SUBJECT 3: (slip indicator under roll pointer and the airplane symbol in the compass rose are marked as 
redundant). 

SUBJECT 4: (a commanded roll carot is drawn on roll scale) - add another “carot” that provides roll info 
during turns. Have radio altitude flash if it gets below a certain value. E.g. 500 feet (with gear not down). 
This will draw attention to how low he really is. 

SUBJECT 5: Eliminate (compass with airplane symbol circled). Shrink A/S and Alt scales. Eliminate hdg at 
top H 255. 

SUBJECT 6: Only suggestion is to allow the ability to declutter some info if the pilot deems it worthy. 

SUBJECT 7: Add synthetic features like runways, towers, mtn tops, i.e. significant obstacles. Change FPV to 
have vertical tail. Add bank angle cues (small roll scale drawn above velocity vector). Add pop up 
thermometer radar altitude scale described earlier. Remove waterline. Add color. Add command steering 
cue (ghost airplane with tail hitch drawn). 

SUBJECT 8: HDG declutter - if selected on: - H255 at top of display becomes larger. MSA readouts. 
Contrast knob to ctl tunnel brightness. If VSD not part of package, have high points depicted with a carat 
or icon. 

SUBJECT 9: Delete GS and radio alt. Add wind vector and distance to waypoint X 

SUBJECT 10: “anticipated path ’’flight director w/ 3 flying triangles in addition or instead of “tunnel”, 
(drawn is a noodle type presentation on the PFD with 3 ghost type airplanes). Move RA out of center FOV 
(the 1470 shown in picture to lower right). Display traditional course guidance LOC & GS (drawn are 
LOC and GS scales on left and bottom of display) 

SUBJECT 11: (circle drawn around compass rose) clutter, could be removed, there are 3 ways to see 
heading in this HUD. 

SUBJECT 12: (noted on airspeed and altitude scales is poor contrasting). This was not the display used in 
the test. Move the heading display down to the compass depiction either inside the circle o on top of the 
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carat. What's the 1470 for? If its radio alt then why is Radio 100 on the lower right? Move the airspeed 
and alt tapes to the outer edge so each would have a black background. This would make more of the 
photo terrain visible and enhance visibility of the light green numbers. Contrast between light green 
number and light green terrain is poor. 

SUBJECT 13: (circle drawn around altitude scale readout) numbers hard to read, (illegible) the altitude 
tape. Occasionally I found the display confusing. Some numbers disappear in the terrain. Tunnel corners 
also got confusing when a/c was pitched up. Corners clashed with pitch ladder. 

SUBJECT 14: I’d take out the lower compass rose - that’s where I’d be checking for high terrain. The rest 
of the items are not distracting. I’d take the pitch numbers off on side of the ADI presentation. 
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14. In the figure, please draw or indicate the changes you would want to incorporate into a head-down SVS display 
or any changes that you think would enhance the performance and safety benefits of the SVS PFD. You may 
draw directly on the figure or use the box below. 



SUBJECT 1: Specifically, Size A, change tunnel color to a muted color. This will eliminate magenta clutter 
on screen. I would like to see a wind arrow in the top left corner of the screen. 

SUBJECT 2: VNA V and LNA V status at top of display. 

SUBJECT 3: Dim the heading components would help (clutter). 

SUBJECT 4: Same as HUD drawing. The flight director does not provide roll info. SO pilot has to guess as 
to how much bank to use. This leads to chasing flight director. Also SVS is a great enhancement but it 
does not provide the “ground rush ” effect if an airplane is approaching terrain rapidly. This is because 
SVS provides limited FOV as compared to real flying. Add something to display that will alert pilot of 
terrain proximity e.g. flashing radio altitude. 

SUBJECT 5: Shrink A/S and Alt scales. 

SUBJECT 6: All these concepts above are awesome. Wouldn ’t change anything. 

SUBJECT 7: Same as question 13. Both should be the same. 

SUBJECT 8: No changes. 

SUBJECT 9: Distance to waypoint or destination 

SUBJECT 10: Add flap bug speeds to tape (ie, F2, FI 5, F30). Change crow ’s feet to boxes and make # of 
boxes a selectable feature. 

SUBJECT 11: looks good. 

SUBJECT 12: Move the tapes off the photo background and give them a black background. This would also 
declutter the photo and make more of it visible. Change the aircraft symbology to a single cue flight 
director. 

SUBJECT 13: (drew circle around velocity vector/ghost aircraft cluster) Too much clutter with velocity 
vector, ghost a/c, heading reference, etc. (drew circle around altitude scale) Vertical speed very hard to 
reference. Gets lost in altitude tape, (drew circle around decision height box) Hard to see in FOV. 

SUBJECT 14: I’d make the tunnel symbology a bit larger; eliminate the vertical lines (magenta) down to the 
ground track X’s. I’d like the scales on the size X to be smaller. 
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15. Overall, please rank order what you believe the SYS display concepts would best support: 


Enhanced Intuitive View and Enhanced Situation Awareness 
Provides day Visual Meteorological Conditions in Instrument Meteorological Conditions 
Hazard / Obstacle Detection & Display 

Displays terrain and obstacles that present hazards to the aircraft as well as provide warning and 
avoidance alerting 
Precision Navigation Guidance 

SVS virtual displays like taxi maps, tunnel / pathway guidance and navigation cues to accurately view 
own-ship location, terrain, etc. and help in achieving RNP/RNAV approach minimums, curved 
approaches, noise abatement, etc. 


Subject ranking of how SVS would support various 
operations 
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a> 

! 5 * 

3 

(/> 

M- 
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SI 


■ Enhanced Intutituve 
View 


□ Hazard/Obstacle 
detection 


□ Precision 
Navigation 
Guidance 

1 2 3 



Rank that SVS display concept 
would best support 


Please comment on your choice 

SUBJECT 1: 1 leads to 2. A picture is worth a thousand yellow and red dots. 

SUBJECT 2: Curved approaches would most improve capability of current system. Obstacle detection not as 
important most places we routinely fly. 

SUBJECT 3: CFIT is the concern of most flying in some domestic airports and many countries out con US. 

SUBJECT 4: 1- Missing mountains and obstacles are critical. 2- Precision guidance is a nice-to-have 
capability but not critical. 3- VMC can never really be duplicated with SVS due to limited FOV 

SUBJECT 5: In IMC conditions getting a peek at the ground is great for SA so a continuous projection of the 
ground or terrain would be awesome ! 

SUBJECT 6: (no comment) 

SUBJECT 7: All very important. Hard to rank. 

SUBJECT 8: Invaluable for # 2 , however would be used more routinely in day to day operations for # 1 and 
#3. 

SUBJECT 9: (no comment) 

SUBJECT 10: (no comment) 

SUBJECT 11: SVS display concepts would support all so I ranked for where I think the greatest need is. 
Already good displays to show hazards. Need to have SA on the hazard. 

SUBJECT 12: Seems like the whole point is to enhance terrain & obstacle awareness whether its on the PFD 
or the HUD. The other benefits are good too but solving the CFIT problem has to come first. 

SUBJECT 13: CFIT is still the biggest safety problem we have. Displaying the terrain and obstacles to the 
pilot has to help the CFIT with the enhanced intuitive view a close second. We are doing precision nav 
now so it can wait a little longer for implementation in an SVS form. 

SUBJECT 14: Managed navigation and automated approaches generally cover normal operations. The 
enhanced situational awareness gives options if fight choices are required. Would provide high 
confidence option if manual/emergency intervention is required. 
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16. Please provide your ranks of what phase of flight SVS would best support: 

Ground Operations 

Departure Operations 

En Route Operations 

Arrival and Approach Operations 

Subject rank of phase of flight SVS would best support 

14 i . 


■ Ground Operations 

□ Departure Operations 

□ En Route Operations 

□ Arrival and Approach 
Operations 

12 3 4 

Subject rank 
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17. Please discuss how SYS display concepts could be used to support each of these flight operations. 


SUBJECT 1: Ground OPS SVS might improve in ranking if proven to be accurate enough with enough 
resolution to show taxiway signs & markings, especially runway entry points, with traffic signal lights 
based on approaching traffic (ground or airbourne). 

SUBJECT 2: Arrival 1 - curved approaches would increase system capacity & help with noise abatement. 
Ground 2 - would help considerably in low viz conditions. Reduce rwy incursions. Dept 3 - not as 
necessary due to high climb rates on most acft. Enroute 4 - waste of money. 

SUBJECT 3: If I know where I am, I know how to get to other points. The greatest concern for me is 
accurate knowledge of present position. In Wx/night/beyond visual range situations. 

SUBJECT 4: 1) Arrival and departure procedures -pilots would have a complete understanding of written 
procedures. 2) Enroute - SA would be improved with better knowledge of aircraft position. 3) Ground - 
SVS could improve taxiing around in fog. Help avoid runway incursions. 

SUBJECT 5: The experiment type display was a good base line for arrival & approach. 

SUBJECT 6: With a high resolution, the limits are endless! 

SUBJECT 7: Arrival & approach is the highest safety threat time having better ground SA will save lives. En 
route is our most benign for airlines so it is of least value. Ground OPS, especially with marginal Wx is 
the second best pay off area. 

SUBJECT 8: 1&2 (Arrival and departure ops) - SID, STAR, approach enhancement with tunnel pathway, 
CFIT avoidance, missed approach guidance. 4 (en route ops) - ifTCAS alerts could be programmed into 
display somehow?- engine out - drift down info critical terrain areas. 3 (ground ops) - SMGCS operations 

- low vis taxiing. Enhanced SA in all flight envelopes 

SUBJECT 9: (no comment) 

SUBJECT 10: SVS would be very helpful in every area except ground ops. Vehicles and other aircraft are 
the obstacle not terrain. 

SUBJECT 11: For ground ops could allow to taxi in very bad weather. I believe taxiing in bad weather is 
currently a limiting factor in A/C departures. In all others provide SA. Closer to the ground the more SA 
is required. 

SUBJECT 12: Arrival approach and departure phase use could reduce CFIT. Ground ops use could reduce 
runway incursions. Enroute use would only help avoid mid air collisions if resolutions is good enough to 
show other aircraft in enough time to allow avoidance maneuvers. Unfortunately, the best, most 
thoroughly detailed display is worthless unless pilots are well trained and take the information seriously. 
Far too many “professional” aviators blow off TAWS and GPWS data because they try to out think or 
rationalize what the system is doing. Until pilot attitudes are changed the best tools in the world won ’t 
stop CFIT or mid air collisions. 

SUBJECT 13: Ground ops - low vis taxi, surface collision avoidance if ground equipment can be included in 
the display. Ground position awareness. Dep ops - better guidance for complex departures, terrain 
avoidance, aircraft avoidance when in high nose up attitudes. Enroute - traffic awareness. 

Arrival/ Approach - closely spaced approaches - improve position awareness, remove the IMC to VMC 
transition during low IMC conditions. Terrain awareness 

SUBJECT 14: Approach - reduced separation during low vis approach ops. Departures - selectable 
departure profiles would greatly aid lookout in congested areas. Enroute - display TCAS items. Gnd Ops 

- low visibility taxi ops, runway incursion -prevention, ground guidance & sequencing. 
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18. Please provide your comments on operational benefits of SVS for the following operations: 

Independent operations on closely spaced, parallel runways (e.g., station keeping) 

SUBJECT 1: Would have to be combined with some sort of ACFT to ACFT separation monitoring & warning 
system with TCAS like (TA/RA). Huge system reliability requirements. 

SUBJECT 2: Allows for curved approaches 
SUBJECT 3: 90 would be helpful in the prm regime. 

SUBJECT 4: Potential benefit depending on pilot workload to process info. 

SUBJECT 5: Good for overall SA but if you can 7 see the other aircraft you still won 7 have a complete warm 
& fuzzy compared with VFR conditions. 

SUBJECT 6: helpful 

SUBJECT 7: Very important - SVS could significantly improve airport capacity and safety, especially if 
artificial view of wing tip vortices could be displayed based on current wind conditions. 

SUBJECT 8: Low altitude holding in critical terrain areas. 

SUBJECT 9: Very good for SA 
SUBJECT 10: NIL 

SUBJE 11: would provide benefits if could be certified. 

SUBJECT 12: useful only if you can display the other aircraft also on the approach. 

SUBJECT 13: could help during PRM to improve position awareness which might limit the descending 
breakout manv. if wake detections capabilities are developed perhaps they could be displayed to allow 
closer spacing to increase capacity. 

SUBJECT 14: Tunnels would enhance ability to maintain strict flight path/approach paths. 

Reduced inter-arrival separation and self-spacing capabilities 

SUBJECT 1: Warning, only with much greater sensitivity. Huge system reliability requirements. 

SUBJECT 2: Don 7 necessarily see how it could help here. 

SUBJECT 3: Opening or closing distances would be self regulated with the better SA. 

SUBJECT 4: Potential benefit depending on pilot workload to process info. 

SUBJECT 5: Not a lot of benefit. 

SUBJECT 6: OK 
SUBJECT 7: Same as above. 

SUBJECT 8: ? 

SUBJECT 9: Only if thightly coupled w/ TCAS in low vis operations 
SUBJECT 10: NIL 

SUBJECT 11: good potential to allow for closer spacing in bad weather. 

SUBJECT 12: ATC has a hard enough time with the arrival separation as close as it is now. 

SUBJECT 13: Same as above. TCAS on the display could help with acquiring a target a/c quicker perhaps 
improving acceptance rates. 

SUBJECT 14: Especially in marginal VFR conditions precise flight path guidance - tunnels - could enable 
closer spacing. 

Converging and circling approaches and departures 

SUBJECT 1: Huge system reliability requirements. 

SUBJECT 2: Obviously, SVS makes complex arrivals/departures easier. 

SUBJECT 3: As proven in the experiment. 

SUBJECT 4: Potential benefit depending on ease in setting up approach/departure in FMC. 

SUBJECT 5: Great SA builder to keep an idea of where the terrain is. 

SUBJECT 6: Absolutely an asset! Low & slow flight, this adds a tremendous safety margin. 

SUBJECT 7: Same as above. 

SUBJECT 8: ? 

SUBJECT 9: excellent but should have TCAS inputs available 

SUBJECT 10: circling approaches -yes - offers additional useful info for terrain avoidance and situational 
awareness. 

SUBJECT 11: good SA builder 

SUBJECT 12: (‘ circling approaches ’ has been circled) in routine airline operations? This will only increase 
the accident rate. 


110 



SUBJECT 13: Improved CFIT - any circling manv. are improved with terrain awareness. Also helps with 
positional awareness when at an MUA altitude. 

SUBJECT 14: Again aids to precision 

Reduced arrival and departure minimums 

SUBJECT 1: Would require photo realistic HUD. This would allow lower minimums for hand flown 
approaches. Not sure what benefit to newer commercial ACFT with CAT III capability, unless it can be 
proven capable of 0/0 landings ie transition to landing on SVS. 

SUBJECT 2: Absolutely! 

SUBJECT 3: Flying either would be aided thru path direction and potential for wake turbulence events. If all 
acft fly the same path, the crosswind would be less likely to allow wake events. 

SUBJECT 4: Great benefit to land or takeoff in lower weather conditions. 

SUBJECT 5: Yes being able to “see” the runway environment will help a lot with arrivals. 

SUBJECT 6:? 

SUBJECT 7: Same as above. 

SUBJECT 8: (no comment) 

SUBJECT 9: Limited use 
SUBJECT 10: NIL 

SUBJECT 11: not really in the commercial sector. Can go pretty low now & I think it's to the point where 
Wx to taxi is limiting factor. 

SUBJECT 12: see comment for reduced inter-arrival separation 

SUBJECT 13: SVS should allow lower mins where ground based aids such as runway lights or approach 
lights are presently required. Departure mins that are based on visibility or terrain awareness could 
possibly be lowered. 

SUBJECT 14: Should allow many more contact approach procedures. 

Reduced runway occupancy time 

SUBJECT 1: And ACFT display. With resolution to allow accurate identification of specific intersections. 
SUBJECT 2: In heavy fog, absolutely. Would allow day VFR landing rates in IMC conditions. 

SUBJECT 3: Flying either would be aided thru path direction and potential for wake turbulence events. If all 
acft fly the same path, the crosswind would be less likely to allow wake events. 

SUBJECT 4: Great benefit for pilots to identify correct taxiway when leaving runway. 

SUBJECT 5: Not much benefit. 

SUBJECT 6:? 

SUBJECT 7: Same as above. 

SUBJECT 8: ? 

SUBJECT 9: Limited. Timing seems best approach. 

SUBJECT 10: NIL 

SUBJECT 11: Not too much with the concepts shown. 

SUBJECT 12: see comment for reduced inter-arrival separation 

SUBJECT 13: In low vis will be able to see approaching turnoffs quicker, keeping pressure on the runway. 

Low vis - help with runway incursion prevention. 

SUBJECT 14: (no comment) 
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Taxi operations 

SUBJECT 1: With resolution to allow accurate identification of specific intersections. 

SUBJECT 2: Sure. 

SUBJECT 3: This is an option that would cause some growing pains. Signage and data linked taxi 
instructions would remain as the primary. 

SUBJECT 4: Great benefit in fog. 

SUBJECT 5: Not much. 

SUBJECT 6: helpful, but unless everyone has it ...then?? 

SUBJECT 7: Big threat are reason controllers are up in towers is to get a better view of big picture, SVS 
brings big picture into the cockpit. 

SUBJECT 8: SIMGCS- low vis - answers the “Where are we? ” question 
SUBJECT 9: very good for pre-planned taxi routes and SMGS operations 
SUBJECT 10: NIL 

SUBJECT 11: Excellent potential & operational benefits. 

SUBJECT 12: see comment for reduced inter-arrival separation 

SUBJECT 13: Lower taxi mins could allow cat 3c (zero zero) if the ability to get to the gate was available 
through SVS. Runway incursion prevention could improve when vis is low. A/c position awareness would 
improve. 

SUBJECT 14: (no comment) 

En-route operations 

SUBJECT 1: Limited benefit. 

SUBJECT 2: No. 

SUBJECT 3: SA, SA, SA - especially for Wx deviation when incorporating Wx on the HUD. Or any SVS. 
SUBJECT 4: Good benefit. 

SUBJECT 5: Good for drift down scenarios. 

SUBJECT 6: OK 

SUBJECT 7: Will provide better view than TCAS will be essential when we go to free flight. 

SUBJECT 8: Critical terrain drift down - engine out 

SUBJECT 9: Limited above 25K’. TCAS inputs on HUD would help resolve traffic conflicts. 

SUBJECT 10: terrain avoidance & enhanced situational awareness. 

SUBJECT 11: Limited. 

SUBJECT 12: (no comment) 

SUBJECT 13: SVS would have to have radar display and TCAS displays to improve enroute ops. 

SUBJECT 14: enhanced situational awareness, depending on the amoun of data points available. 
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Aircraft energy management 

SUBJECT 1: Limited benefit. 

SUBJECT 2: Not that I see, but I guess in a IMC low energy state (stall/windshear) having a ground 
reference could help. 

SUBJECT 3: Fuel savings would be tremendous. Proper spacing and reduced vectoring by AT C would 
enhance both SA and fuel expenditure. 

SUBJECT 4: Good feature. 

SUBJECT 5: Overall SA enhancement. 

SUBJECT 6: Good idea. 

SUBJECT 7: Big pay off here if implemented right. 

SUBJECT 8: (no comment) 

SUBJECT 9: Limited use w/o AO A info & Ps diagram. 

SUBJECT 10: helpful during low level ops 

SUBJECT 11: Limited. 

SUBJECT 12: Airline management will negate any possible benefit because of their inability to plan 
appropriate profiles. 

SUBJECT 13: Projection of a path (tunnel) that is base on a/c energy state might improve fuel economies. 
Fix ETAs for A/C management and CFIT reduction if projected path to the ground is displayed. 

SUBJECT 14: (no comment) 


Other 

ONE SUBJECT RESPONDED: The aids to SA could be very important in unplanned go-around situations. 
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19. Please comment on the use of SVS display concepts for supporting non-normal operations and discuss how 
SVS may benefit these operations. Based on your experience during the experimental study, please discuss the 
use of SVS display concepts for non-normal situations: 

Some Examples: 

Upset recognition and recovery 
En-route diversion planning 

Traffic and weather hazard deconfliction during engine out drift down 
Improved emergency descent awareness of terrain, traffic, etc. 

Depiction of missed approach guidance 

Intuitive emergency procedure support and guidance 

Others 

SUBJECT 1: HUD & SVS would almost eliminate possibility of tunnel vision (decreased scan) due to 
abnormal operation or emergency. 

SUBJECT 2: Greatly reduces pilot workload during IMC arrivals. If pilot not flying is working emergency, 
allows for safer operation by increasing pilot SA. Great aid for upset recovery. 

SUBJECT 3: The possibility to flying approach (practice) at altitude prior to descending into either 
mountainous terrain of foul weather could easily prevent CFIT events. 

SUBJECT 4: SVS provides valuable info such that if a pilot loses an engine, the pilot can determine which 
way to turn the airplane to avoid hitting terrain. VSD is a good tool with regard to this. 

SUBJECT 5: Engine out terrain avoidance both on T/O and missed approach would be awesome. Also drift 
down terrain avoidance would help also. 

SUBJECT 6: Help to reduce pilot workload during high stress situations. 

SUBJECT 7: Only non-normal we did was eng out but this is a big, big benefit in all examples mentioned 
below. 

SUBJECT 8: Enroute diversion to little used divert fields - unfamiliarity with terrain - drift down in critical 
terrain with escape routes - Alaska, Europe, S America, Rockies, etc. 

SUBJECT 9: Adverse weather (thunderstorms /windshear) would be a big help. Upset recovery also a great 
benefit. Have you considered drift down representation? 

SUBJECT 10: Extremely useful for terrain avoidance for emergency descents and engine failure situations 
during low level and/or high terrain operations 

SUBJECT 11: While is no necessary to determine a non-normal situation is occurring, it was essential in 
providing guidance in what to do during a non-normal situation. 

SUBJECT 12: (no comment) 

SUBJECT 13: Altitude awareness during non-normals might be one of the biggest advantages of SVS 
followed by terrain awareness. Both of these are concerns during engine failure non normals. 

SUBJECT 14: Coupled to TCAS - could be important to near-miss/midair avoidance especially in 
arrival/departure situations. Terrain proximity awareness could be the cure for many distraction-related 
potentials. 


114 



20. Please comment on your confidence in using these SVS display concepts to make CAT IIIc approach, 
departure, and taxi operations 


1 

2 

3 
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00 
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Not Confident (0%) 
Very Confident (100%) 


Subject's confidence rating using SVS display concepts to 
make CAT IIIc approach, departure and taxi operations 
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100% 90% 80% 70% 60% 

Subject's confidence rating 


SUBJECT 1: 9 In APP & DEP 

SUBJECT 2: You don ’t hit the ground when you can see it. 

SUBJECT 3: Actual NAV performance displayed versus required NAV performance would this a 9+. 
SUBJECT 4: All of the visual cues are present with SVS to perform these tasks. 

SUBJECT 5: (no comment) 

SUBJECT 6: (no comment) 

SUBJECT 7: (no comment) 

SUBJECT 8: (no comment) 

SUBJECT 9: (no comment) 

SUBJECT 10: not useful for taxi operations because it offers no information on/about obstructions vehicles 
or other aircraft that maybe unseen. 

SUBJECT 11: Should be very good as long as it is reliable. 

SUBJECT 12: Never done CA T IIIc 

SUBJECT 13: Biggest concern has more to do with data integrity and accuracy then SVS capability. 
SUBJECT 14: 1 think superimposing the guidance on heads-down systems would greatly improve planning 
and situation awareness during the approach. 
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If not 100% confident, please comment on what could be added (e.g., sensors) to increase your level of confidence 
SUBJECT 1: No exposure to landing and taxi operations on SVS. 

SUBJECT 2: For Taxi OPS, some type of sensors to keep you from hitting other aircraft. This SVS will keep 
you on the concrete but you can still hit anything not in the terrain database. 

SUBJECT 3: Actual NAV performance displayed versus required NAV performance would this a 9+. 
SUBJECT 4: Ground equipment to aid in aircraft positioning. Expanded FOV to 180 degrees would aid in 
ground movement. 

SUBJECT 5: NA V error alerting system. Does the A/C really know where it is or does it think it knows? 
SUBJECT 6: (no comment) 

SUBJECT 7: (no comment) 

SUBJECT 8: (no comment) 

SUBJECT 9: (no comment) 

SUBJECT 10: FLIR and MM radar views ie. Real time info systems independent of a computed database. 
SUBJECT 11: ensure data matches real world. 

SUBJECT 12: (no comment) 

SUBJECT 13: until the data integrity issue is resolved I couldn ’t be 100%. 

SUBJECT 14: Add realiabity /confidence builders - how close is computed position to actual? Differential 
GPS? 
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21. Please discuss whether SVS display concepts could be beneficial as a flight planning and “mission rehearsal” 
tool (e.g., rehearse an approach while in cruise through one of the multifunction displays). 

SUBJECT 1: Only if reward is ’’top gun ” video played on Size X after successful practice approach. 

SUBJECT 2: Never thought of that - could be a great idea! 

SUBJECT 3: Refer to 19. 

SUBJECT 4: Very beneficial. 

SUBJECT 5: For special qual airports, complex missed approaches, complex engine out procedures the 
system would be a great practice tool. 

SUBJECT 6: Absolutely, especially in hazardous airport operations like Vail, CO. 

SUBJECT 7: These could be adapted to home computer use and allow pilots to train themselves to a higher 
level than today ’s standardized training programs. Big $ savings here too while improving training. 
SUBJECT 8: ? 

SUBJECT 9: Could be useful on flights > 1 hr. Photorealistic must represent current conditions [sun angle, 
moon, diurnal and gnd cover (snow, etc)] 

SUBJECT 10: Good tool for mission rehearsals on the ground-simulations. Impractical for use in flight. No 
way to manipulate simulation controls. However, it would be very useful to be able to run a “preview” of 
what the expected approach will look like - in flight. 

SUBJECT 11: yes, currently being done in military 

SUBJECT 12: 1 believe SVS displays would be extremely useful in special airport qualification training 
required by FARs also for refresher training when combined with outside view & simulator. Specific 
difficult approaches to short runways on those in mountainous terrain. 

SUBJECT 13: Flight planning for route verification might be a use. I have never had a need for that example 
of rehearsing an approach in cruise. Rather than rehearsing per say. It could be used for a “visual ” 
approach briefing which is now done verbally. 

SUBJECT 14: One could certainly benefit from it as a briefing tool - show 3D depiction of approach for 
comparison to approach plate. 
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22. General Comments (or attach additional sheets): 

SUBJECT 1: My HUD experience suggests average pilots can fly precise CAT II or lower approaches. The 
addition of SVS adds a huge amount of capabilities in terrain avoidance. If landing transition and ground 
OPS are proven (if not already), then you have taken the next big step in aviation safety. 

SUBJECT 2: Overall, a very impressive system. General comments: Size matters - large display is better. 
Ghost airplane - very nice concept - easy to use. Symbology -just right - don ’t put any more clutter on 
HUD but no need to remove anything you have 

SUBJECT 3: The HUD allowed me the greatest snap shot. I could fly the HUD and quickly scan the 
instruments to confirm. The X size was also good however I more easily adapted to the HUD. 

SUBJECT 4: Overall, I think SVS is a great concept that needs to be implemented in the cockpit. I have some 
concerns though: First, in the simulator that I flew for the experiment, the approach and missed approach 
were already set up for me to fly, so the workload was relatively minimal. My concern is with the difficulty 
for the pilot to set up the approach: are the approaches in the database already? How easy is it for the 
pilot to change approaches at the last minute due to ATC request? Not all glass cockpits have approaches 
built into their FMC database. Regarding approaches that include visual maneuvers to the runway: the 
approach flown in the simulator had a right turn to line up on runway 7. This approach requires the pilot 
to be visual with the runway during the maneuver, but with the limited FOV in the simulator, the runway 
wasn ’t in sight until the final turn. ATC expects the pilot to maintain visual with the runway during the 
entire maneuver but SVS doesn ’t permit this. One solution is to increase FOV. I suggest SVS be used as 
an enhancement to straight-in approaches only until FOV can be expanded. Final comment goes back to 
pilot workload. It doesn ’t matter how great SVS is if using it or working with is too complicated. If a pilot 
has to spend several minutes setting up his display for the approach then it is not an enhancement. If it is 
very easy to make a mistake setting up the approach then it is not an enhancement. I suggest focusing next 
experiment on pilot-SVS interface during approach set up. Have pilots setup several different approaches 
and evaluate them on speed and accuracy. 

SUBJECT 5: SVS would or will be a great situational awareness enhancement tool when it is IMC outside. 
The guidance cues will make many difficult engine out, missed approach or arrival maneuvers much easier 
provided the box is programmed correctly and the aircraft NA V is right on. 

SUBJECT 6: Certain airports which seem to rely heavily upon visual approaches such as River visual into 
DCA and expressway visual 31 into LG A would be easier and safer with information such as what we have 
here. Especially in marginal VFR conditions it may serve as a backup tool when outside factors are 
present, such as at night, hazy weather or inexperience in operations to that airport. 

SUBJECT 7: Continuation of question #2 - salient features = flight path vector (marker) with speed worm 
and energy carat are the most essential displays - they should be combined with a good steering cue like 
the NASA aircraft but when a turn is commanded the bank angle required for the turn should match the 
lines See diagram at left (drawn is a banked velocity vector with a ghost airplane with a hitch ball as a 
target. The wings of the velocity vector and the base of the ghost airplane are parallel which gives 
roll/bank guidance) - these diagrams are a slight change to the NASA cue in that the command circle that 
is placed in the center of the FPV is mounted below the bottom of the triangular steering cue. Pilots like to 
pull toward a cue and it also keeps the FPV symbol off the top of the steering command cue making it 
easier to watch the command cue. 

SUBJECT 8: (no comment) 

SUBJECT 9: (no comment) 

SUBJECT 10: (no comment) 

SUBJECT 11: (no comment) 

SUBJECT 12: (no comment) 

SUBJECT 13: (no comment) 

SUBJECT 14: 1 personally prefer the generic for all presentations. Photorealistic is pretty, but could be 
confusing if the “photo ” isn y t realistic (ie, snowing? Dark or light?, etc), with a generic display you know 
it's a cartoon with salient features highlighted i.e. - blue river/stream, brown high terrain with a “ green ” 
valley /flat terrain, a few black shapes for buildings, etc) I think I’d transition more quickly from generic to 
actual than from “photorealistic ” to reality. I thought the potential for these concepts to be limited only by 
the imagination of the user. Depending on the confidence level in the system used for determining present 
position, computed guidance and computed terrain modeling can essentially put visual flying into the 
PFD/ND. Collision avoidance, altitude excursion avoidance, crossing limitations; anything that can be 
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programmed or receivd from outside sources can be projected + improve SA. I’d be interested in seeing 
multiple, selectable approaches flown using SVS as compared to pilot performance using approach charts 
and NA VAIDS. Touch screen selectors for options and ranges would add ease-of-use. I’d add higher 
definition to the horizontal EGPWS depictions. A database that provides photo realistic or this level of 
generic display should be able to show a better “anti aliased” depiction of EGPWS data. The opaque 
nature of the HUD seems limiting to me. Could some sort of translucence be included? Maybe use a 
“scatter-dot” projection (does that make sense). Being able to see actual terrain through the projection 
would be confidence building. Ed recommend playing with different options for depicting the tunnel; 
maybe add some dots connecting corners of the path as shown on the screen. A bit more density might 
make it an easier reference. For the departure, a tunnel probably isn ’t appropriate but a simulated floor, 
maybe showing min terrain clearance altitude over the programmed ground track might help (or am I 
forgetting something you already have?) 
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